et 


F Ss 


fg a ty, "ty, 4 9 ty ES g 
| . ss * OCeNRNS BESUNE 3 an 
| 14 o a * " ’ : Ar + ane 
~< BD 182 082° Loy ea Ps ori 198° 
ae oF - 4 
. TITLE ‘Waking Innovation work: A secead Year. Report on Title 
: awe . IV-C Projects in Magsachusetts, 1977. 
“INSTITUTION: Herrimack Education Center, Chelasford, asa. | 
‘SPONS AGENCY... Hassachusetts State Dept.-.of. sah th Boston«- ae 


POB DATE 


Aug 1 2 ie Xe ‘ U 
NOTE. 1 iy 


168p.; Best copy avai lable. ee 
t - . . 
| BDRS PRICE aFO1/PC07 Plus Postage. 
‘DESCRIPTORS .- Administrator Role; *Diffusion: “ wpducational 
: : : ‘Innovations Educational Progranas;- ‘Blesentary : S28? 
‘a Secondary Education: *Pormative Evaluation; © 
pe Guidelines; Objectives: Problea Solving; *Progras 
' Evaluations *Prograns Taprovenent; pacgete Validation: 
| Quest Lonnaires 
./\#Blementary Secon dary paneer son Act Title IVs 
‘| whassachusetts: 
ee nit , a be 


This. report, the second of three prepared for the 
_fHassachusetts Department of Education, focuses on a formative . 
‘evaluation which identifies descrepancies between inncvative Title 
* Iv-C. educational programs as flanned and as the prograas operated. 
“Contents are organized in five main chapters each of which represents. 
| a different perspective but all of which have a common purpose:. to . 
illuminate the change agent process at mid-point. Chagter One : 
provides a discussion of the four project types: experien 
curricnlar infusion, special needs and central service. T 
assistance role of the Educational ‘Research Corporation is dey 
in Chapter Two. Chapter Three contains guidelines for designing. ‘ans 
evaluation process. Chapter Four @ves a complete example of a : 
~walidation report. The final chapter offers a sinulated version of a 
walidation report which states four objectives and-shews how data is 
collected and treated to produce a set of findings and conclusions. 
The Title IV¥-C ESEA Evaluation interyiew questionnaire, the project 
director questionnaire, raining session agenda, 
results, and guidelines for aaing. atanderdszen tes 
(Author/R#) 


oe 


 SEDENTIPIERS - 


¢ 
’ 


~~ a4 
fusion, 
technical - 
tibed — 


sts are’ evaluation 
s'are appended. «-' 


as 
Mg ea 


PTrettittitt tiittti tit ittt titi it ttt SEKREEKE SEES HD OAHEAEKEREREKEKEE EHEKE EEK 


‘* Reproductions oe by EDRS are the best that « can be “et _* 
; rom the original docusent.: 
; cunndanaennennennnnnannssaegsaneannns seannnnnennn se enesenenee, seeeeees 
ie ? : 
e 7 
' am > 


uc Fe ey 
; t. P i aR 2 imme en a : { \" ‘ 
iy F - d " ’ ra at bs 
c U 5. DEPARTMENT OF W a4 i 
Te eALTH, =| 
ie . . 3 GOUCATION & WELFARE pe’ 
WATIOWAL INSTITUTE OR , } ‘| 
« .  |oucation Te re We 


a4 = : 
THIS DOCUMENT HAS BEEN ; 
REPRO- . 
Ouceo EXACEAY AS RECEIVED ROM 
He ERSON GH ORGANIZATION ORIGIN: | ; 
ay NG IT POINTS'OF VIEW OR OBINIONS 
ATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 
| EDUCATION POSITION OR Pouch 
J 


- 


ee 


ED182042 


oe sats ‘¥ Fr Peer ; ee é %, 4 
; MAKING INHOVATICN WORK , 
: ~ a ae Pept . 
: : 7 ea Hm 3 A} hy Li. 1Y% 
eg oy ARTS 7 
. C? = f Hi | ‘ : 
o “" ib , one ’ : 


] ‘ : . bg te ee, ; st re . ‘ys 
ey 4 r 5 as 1? . 9 - ° ¢ 
~/ A Seconp Year REecrt on Tite IV-C Provects 


_ IN MasSachuseTt$, 1977.” | : 
; | Pn | 
v 1 / | Ce 
BEST COPY AVAILABLE 
. | . » : | : ‘ & 
Pam o SUBHITTLO TO: 
| Dr., John F. Reynolds. 
Division of Curriculus Services 
ESEA, Title IV-C ~ si 
Massachusetts Deparument of 
Fon . Education 
\ : , 
, era ys cae 
CO 1 a # { “PERMISSION TO REPRODUCE THIS ; é 
; ; * e MATERIAL HAS BEEN GRANTED BY 
(op ae ig 3 Merrimack _. ir Ss 
. » baal (a 4 S j &; wy 1 
tM ( BR tty 
‘ yen “Ny . Hi RESOURCES 
; rn Pa . : ein vee 4 TO LSU CANTER (ERIC).” 
¢ | f f 
a ye te * , 
| ee Merrimack Education Center’ che By} 
‘ OL M11 Read * d 
op) , Chelmsford, Massachysetts 01824 ' an . &. 
et (August 1977 Pde og 
* ) ; : 
EB r 2 Vv. 2 Mi : e "os “ 


/ . ret 
i ft tw . F ‘ 
: ? “ : al 
~ = % , a 
.. ey 
us a " : 4 . , : tt ’ é * 
HASSACHUSETTS DEPARTMENT: OF EDUCATION ~ = ** 
eRe ‘ e Fi ce er - 
ESEA TITLE IV-C  - 
¢ : ! i ¥ . . 4 ‘ 
‘ ADVISORY: €OUNCIL MEMBERS —P 
s ; é , . , r 
Jean McGuire) Chairperson Betty Ann Limpert 
‘ ‘Raymond Almeida Inabeth Miller 
* oe George Banks Nicholdés Muto 
- Rosemary Barszcz ea Neal 
Margaret Brown , ‘Wayne Peters 
, Janet Cook * vito Sammartano - re a 
Gerald Diehl . John Simoncini ' . 
Martin Feeney — (f Joanne Wood 
Francis LaPointe ? . y 
. ~ i I. 
ESEA TITLE IV-C STAFF i: 
\ " 0. ‘ 
“ spa - Judith Dortz 
Maria Grasso 
William Moloney . 
Charles Radlo : 
Ellen Sarkisian a] 


‘Ay 4 . Albert Trocchi - 
“"\, \° John Reynolds, Coordinator 


e 


EVALUATION TEAM 


NSN 


Leslie C. Bernal Richard J. Lavin 


Merrimack Educaticn Centor: 


Allan Ellis 
Educational “Rescarch Corporation. 


Ronald G, Havelock % 
Merrimack Education Center: 


vw 


' Merrimack Education Center - 


doar, E. Sanders < 


Merrimack Education Center 


Pamela Woodroffe 
Educational Research Corporation 


Par) ’ fT we 
ry ¥ ' - 
f i \ 
2 . . ’ 
i oo ‘ - . 


0 ee ore “TABLE OF contents Be ar ae 
we) aio eS , 
rar 4 Ts epee = I eekivapounuclisalnsi eat oo cE tee Ghat dageade Say 
es e . me oi ; ; 
) Pa , of i 
c Pf eh b A ' : 
Fonewbnn : PG, ite: “3 7. take i . 
. Richard J. tavin | ‘y ‘ a ay Me ass : yf 
prawn . gabe, ot BB. " ea. 
“se *” Nerrimack Education Center Staff. ° % ty : ; > ae f 3 
: F i ; : = ; 
| ~ LEARNING THE InnGvatave Poses BY Done. « ae. ee 
Ronald G. see Ck (ese in SAS Mes 
THE TECHNICAL SSISTANCE RoLe OF ERC © Me ond 41 - 50 ‘ 
“ Reneat sans: nie Corporation ae . c ' 
| unde tines’ FOR DesiGnine a. EVALUATION Paacess pe 51 - 6h 
4 - Pamela Woodroffe’ of ERC eae 
eae rukon aaoanien ge. ently | 66 106 
a . Merrimack Education ‘Center Staff . OP tee 4 . 
s : se. ee Oe ) | 
A SIMULATED. VALIDATION REPORT - "407 = 122 
Merrimack Education Center Staff .— ; oe ees 
AppenDIcES = iS . = -, 
“ . “t »\ ‘ ‘\ 8 a iy i , . a : 
SELECTED, READINGS ON EVALUATION AND WRITING OBJECTIVES 
5 . 
‘ 
t 
r ‘A 


se 


' 


fs ay 7 “yi aA “s : : 
7) y vif 

x 3 fee 4 

Since Funds for tihg on vi Were. ie ‘available to education sta the : 

passage of the £1 paitary 'y and Secondary, Education Act, of 1965 -(P.L. 89-10): 
and the’ 1974 ext isi, n of the ESEA Act-(P.L. 93+380),' large amounts of ° 
dollars have: beet’/iinvested through tne States to administer programs to. 
advance education.’ The implications for. evaluation--evalustion of cOst- - 


effectiveness, effectiveness of goals and objectives,-and of progress . 

bang made towards .the achéevement of innovative goals and Objectives-~ 
re great. Over the years many reports and studies, such as the Rand 

Report and the LONA@STEP Report, have been made available, and profusive 

literature on educational change has developed. Yet, twelve years and 

- .a billdon dollars later, we know little-more than we did in 1965 about | 
‘what really makes innovation work or how to diffuse Tnnbvations. 


litle III and Title*IV-C have funded’ many local: innovations. in Massachue 
’ setts.. As new programs are developed *by, Tocal. school personne] through 
Title IV-C the Massachusetts Department of Edu&ation has embraced the ; 
concept of evaluation as the way insure that'public funds tesult in 
_ impact and educational gains. One of the major responsibilities of the’ 
Massachusetts Department of. Education is to assist the schools in.evalua« 
ting program erreoeivenhss top ABNGRAEINE: ‘projects. * _ why 
The process of change, as ‘anyone o has been involved with educational . 
innovation can attest, is extremely complex. ,Yet through the “examination 
of educational change there are a number of insights that can prove very / 
useful to the practicing project administrator. In this examination of / | 
educational innovation in Massachusetts, MAKING INNOVATION WORK, we have/ 
reported:on how people are dealing with the questions of how.to create / 
. innovations in the schools, and at the same time we have worned with # 
e.. in their local. een ra - vs 


¥ 


“‘ThYs evaluation 


howned expert in- the field of POULT and cana amectade 

and Glen Heathers, in the area. ad educational innovation and indjf 

ized instruction. ‘Local insights have been''developed through the 

cation of questionnaires and through onsite& interviews and meefings with 
project directors. This mix of meetings, the literature on Change, 

“” contact with people, ae ne arERS interviews has. provided us with the | 
enclosed reports f 


In this report: Ree ee for thie: ‘ipaaseniawtes Department fs Education, 
we have focused on a formative evaluation which identifief discrepancies 
- between the programs as planned and the programs as operfted, The’ 
formative evaluatton looks, for actual or potential prob#ems in the opera- 
tion of Title IV-C projects in order for adthinistratorg to make adjust- 
ments of refine the scope of: their projects. This refort does not pass’ 
qualitative judgments on the projects; it simply is #'means of gathering 
ATROEREN OG that reflects the. progress and charactey of the way Tal IV- C 
7 } 


ts bein toihocaniel in Ae ‘information that can he aie iz 
'“&to modify the. design of existing projects of to plan new and me ak 
~shouTd prove of enormous, benefit to the ‘Title IV- fs Adyisory- ,_ ee = 
HAKING NWOVATION sWORK js a document designed to be of ‘*  - 
ee Department of: Education fm a gl? AY 
staff, and the Advisory -Council for Title IV-C as they.move in the ~ . 
direction, of Laeraaasli and: git successful. practices. . - 


project. directors, 


 *In-the past two years, Herr imack Education Center! s efforts have been to 
- focus on ‘two areas. of evaluation a@d.administering change ‘through ' 

1 | -information and training directed to project“directors, This evaluation 

design has” bean developed specifically for: the purpose of providing. . -: \_ i 

project direct6rs an evaluative perspectiye in administering their © — i ey 

Title. 1V-C innovation programs. There fas been a deliberate attempt 

- ‘to help ‘these project¢ as they formulate their respective plans and et teas 

= program operations. This report, MAKING. "INNOVATION WORK, is thef result ane 

of somé lessons learned from these gt tte experiences, and @s such, 

it reflects ideasjon some of the mat effective and broduesig evalua~ 

tive arrangements . 


The Merrimack ‘Education wee bat, shateé to ‘cooperate * with the 1, 
Title IV-C funded) projects in this evaluative phase of their efforts. © ‘ 
-. During the. past two years, the EC evaluation team has, had the oppor- ee 
_, tunity to meet with project directors in groups and also to visit,the - - : 

. sites and talk with project directors ‘individually, about.their efforts be at 
on behalf of innovation in education.~ We would Jike to compliment the 

project directors on their etforts to date in providing innovation jin | Ne: 
‘the local schools and their desire’ to make innovation work, since. we 
kn full well the piCiieul ey that this progess entails. — é 


the eniphasis today tn ‘looking at innovation. is a focus on evaluation 
and valigation/as the State adopts procedures. for, evaluation and identi 
fication of Bi worthy of replication by other districts.+ The new | 
validation process, based. largely on the review: process of the JDRP. - 
‘ (Joint Dissemination Review Panel) will-identify projects which Success- 
fully address inetrucksanet impact -in. the schools, This results approach - 
has emanated -from the ree tea of Health, Education, and Welfare in 
* meebingwen in: their efforts to justify-innovation to Congress ‘and, in. 
ddinhg so, to bring successful practices to the local, user... The efforts 
by the Departnent_of Education, Division af Curriéulum ‘and Instruction, §. ,' 
,and the Tits IV-C Advisory ‘Council should offer ‘a sharper focus on the 
validation of: these. efforts in ia next two years: of et funding « 
for these projects. : : 
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errinaek Education Center is "éngaged in Z ‘three-year ‘project to provide, 


with the ‘Massachusetts Department of. Education, . leadership, Support _ 


. and technica] services to twenty-four’ ESEA Title IV-C projects in the — 


Bay State. The imposing purpose of: the project -is not solely to hel 
assure the success of.each project. but also to develop the capability. of. 
diffusing, these projects. Simultaneously, the Center.undertook to 

‘ render similar assistance to BLEED pene ae ee dla 
across the State, , 


Mi te 


Thus, our progress report is the second of three and the one -produced, 

at mid-point... Last year's report,, EVALUATING ESEA TITLE III PROJECTS‘; . 
- IN MASSACHUSETTS, 1975-76; included a: description of the first. wear and « 

made six recommendations which, focused largely on traifing needs. That - 

first-report proved helpful in» designing a fra work for the’ VHT 

activities accounted for in this Second repor 


i lea 


» 


ror) dable. Ine 961, -C.P. Snow could write: i 


' Ina society like ours, academic patterns change ‘. 
more slowly than any others. In alifetime,,in  , 
England, they have crystallized ather than loosened. 
I used to think that it would be! about as hard to + 
change, say, the Oxford and C aie scholarship 

, examination as to conduct a major revolution. TI. 
now believe I was ‘over-optimistic. 


is = 

“In “appreciation of bas BS oblems inherent in’ ‘Anioation and in diffusions 

“ MEC and the Departmen linked to render technical Boat Stance. and 
sali toi a new crop OF prcrrt sin, programs, red 


WHAT: IS PRESENTED : 


Our reeorl serves aS a progress report, a stricturh for reflection 4 
a basis for action. The issues are discussed by Dr. Ronald Havelock. in. 
his Chapter. Promise and peril--it' S all there.- 


o: 


Mach” chapter in our report: represents a ‘different sereiectlve but with a. 
common purpose: to illuminate the change agent process at mid-point. ° 
The chapters. have been edited to assure clarity and a tight presentation. 
Yet each author's style ts intact and 28th author has ‘shaped gis material 
to serve a a spagitic purpose, ; 
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, We at MEC are curnentl¥ rin the “second year of this thheesyear program. } ° 


& 
« 
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ae general ‘aitalysts of approaches and events is described by Havel ck” 

ha been invol ed with the’ project from its inception. .He offers : 

Saree . nN organizational framework for assimilating the impressive siount sea ae 
ety ee OE ha ge ected under, his tutelage ina series of structured interviews. \ 


“To show the Havelock buarotch and to note for the reader the oractical Lee.) 
; » . perspective ‘of, those working on the pfogecty the interview materials E Sus 
 “wncluded. this. rationale: “* ; ‘ # a 


There aie three objectives to these field visits wen + FR aN 
are Mmking-now. The first, and, perhaps most important, b. 
_is to determina ways in whtch. we might.help out wi on ny. 
_various phases Of the project, especially with issu on y 
2. | 2 . long-term maiptenance and dissemination. saccaiys 2. ee 
. . We’ want to get: seme information from each of the projects . |( ‘+ ° " 
a bnich we can use later in the workshop’ sessions we\.have PP wa 
“remaining. The third is to tollect information for the ~ y° 
State on the'problems and progress of this first year | . ; (9 
of Title IV-C which will- help the State sta¥f do a on Fa ot 
_ better job next year. ; a ca { 


Overall, as Havelock reports, efforts to discuss the In procticon on a one- 
to-one basis or in project Chapters appeared : to predice abetter working = = + 
relationship. fi ate 


Havelock allows that ina from year bne to year two is ry Bs means | 
easy to define. It is clear that ‘no tidy and obvious sequence of . _ 
achievements 7's apparent--which is not to say progress is not taking © 
place. { 


. Havelock ends nie ehantal by strasetng: the importance of process. He | 
recommends several different strategies to MEC in the coming Final 
‘year of Title IV-C funding. ; 


~The Educational Research Corporation (ERC) chapter. serves three specific | 
and useful purposes. The first is to compile information about the 
1976-77 Title IV-C projects in the Bay State with an emphasis,on project . Peers. 
evaluation. The secon@is to use the data to suggest ways to proceed _ 7 
be in the next yeariof activity. And the third purpose is to#render an: 2, ae 
é accounting of the work of ERC relevant to" this project. ee, 


. - 


Through the second purpose, ways to proceed in thé.next year, the’. 
Pal! 4. formative evaluation processes are swung into a state: of readiness. ms 
af _Yince both project success and project diffusion are made one and the . 
ae ame in Title IV-C programs , the uses of interim evaluations such as oe Y % 
@ 0 be powerful, 


A T development and the diffusion of a project typica ly waniiteee that ~ 
ap lication of a‘variety of high-order skills, and sometimes felicitous . 
circumstance. Not thei least of these skills, outlined later in the 

ERG report, are (1) a lear Understanding of the project and project 


ay 


a 


covering ERC's IV-C training 


» 
. 
ay 
"aston, 
~- 
a 
. 


‘ea: 


‘objectives; (2) an “effec ive system a Goan iadn: (3) an ‘operational: 
_ process to bring of f epee ands. (4) a battery Bf. ways sa evaluate - 
what's happening. - : ie ae 


. 


ERC works its intent by’ using two ‘kinds of instruments, “the: fitse is a le ae 
Project Director's Survey, ance second invofves the evaluation TOrMSs \ 3 
ssions. / , 


t 


~ General concensus” snd easy agreement are not hallmarks of > the 1976- iv dl 
‘IV-C project director's responses to the survey. However, most did : uy 


i agree that the projects had ‘satisfactory beginnings--that pace, 


facilities; staff and the like were available. nee mos t who renpaneeg > : 


. were: positive about ERC's graining sessions. 


’ In Chapter III. Pam Woourof fe of: ERC furnishes us with a etcaty turned’ 


‘Ms. Woodroofe stresses the need to dévelop a se 


process which is: presented | is both comprehensive and det ied, 


™, 


yi ee * to problems must be balanced with the-sharing of 


4 


out evaluation process: ‘complete with do's and don't's and examples. 

Her conviction and ours #s that evidence must be supplied showing the 

efficacy of: adcpting an innovation 4s part of we diffusion process. * 
of: ‘objectives which : 

can be ‘tested, Her iptroductory materials are sunplseantat be a selected ' 

11st,’ of reading found at the end of this report, | 


The final twol chapters hetp to tell the story of aariaelin iy a ty Gita: 


practical way. The prospects of adopting any ‘innovation that isnot’ -, 4 rs 
evaluated and ‘not validated must be'viewed as bleak. The. val ee ; 


Ih summary, the. 1976-77 ‘year: has, seen its share of diattanse and of ie? 


-prdgress on substantive issues, The workshop leaders and their topics’ 


were Bhesen in responsé to the needs of project directors. ar 


“Jack reynolds, Title IV-C's: Coordinator_in Massachusetts, laid out the - » 
. ultimate Set of ‘issues for the projec when he wrote: ~ eas 


A major :facet. of ESEA, Title IV-C ‘is the identificas = rer 
y +tion and sharing of effective educational programs, 


“ic ‘tig. Phe pressure for more “new, programs has clearly been Ma ‘ is 


‘joined, if not replaced, by a cali for proof of * aes: "| 
effectiveness. Development of innovative approaches 


successful practices and_thea termination of inef-: ; es 
fective programs. Educational agencies at all levels; ©. SS gye 
Federal, State, and lo¢al, have an increasing obliga- : “a 
tion to provide’ thorough evaluation of programs, ¢.° * y . 

“ This must be done so that informed decisions can’ be ' : 

. made as to\whether the program ghould be continued, ‘, io 

A expanded, or terminated, Funding. agercies must pat 

a “consumer ' protéction" role, through program evalua- 

tion, in determining sthe’ effectiveness of the 

peegnaee they fund, , ° ‘ ue eS 


. What remains is that this report be studied. by those who care about 
innovative projects and thetr diffusion. Yo study this report is to 
prepare ourselves. that much the better for the SACP and. cracaall: ” 
year ahead, 1* 
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LEARNING THE INNOVATIVE PROCESS: BY DOING. 
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_ The 1976-77 school year marked the return to. an caaniter policy for, 
federally funded and state administered support of local innovations in’ 
school districts. This year's effort consisted of thé expenditure of- . 
nearly $900,000 through twenty-four projects, with the promise'of - © «\ 
affecting up to 75,000 students at least indifectly and perhaps as many 
as 2,000 directly. . Most of these projects were new starts.although some 
had benefited from modest planning grants. and others could be. described © 7 
in etl spin-offs\from previous efforts in different districts ~~ ns _ 
inside atid outside pearihe setts. Project applicants were also encouraged 
to plan in terms of a three-year funding cycle leading to the full . 
acceptance and: integration of the inndvation within the local district 
thus reverting to a policy suspended two years es as the old 

Title III program of ESEA wound down. 


* 
ais 


In the last year of. Title III the state had Giana tan Aiea 
projects on a One-year. planning cycle and had invited Merrimack Education 
Center (MEC) to provide a sable Hs gs of formative evaluation and - 
training for project directors. h the training and fhe evaluation 
focused not so much on. either the content or expected outcomes of the 
projects as on the-process by which ‘they were planned and managed. ‘ 
Results were summarized in a report for MEC by Dr. Havelock entitled; 
"The Process for the Product." »In brief, it was noted that project - : 
directors come. to their jobs with considerable enthusiasm for the asia 
ment of education and:-dedication to the particular type of reform -_ ~. 
which they have proposed. Nevertheless, they are frequently frustrated 
by ‘the complex ane often seemingly conflicting demands of their own, role. 


The Havelock-MEC report of 1976 was strongly confirmed by a follow-up 

study by Widmer (1977) on ne ele III. projects in Massachusetts — 

funded in the cycle, 1971-73. ies previous study of the same projects . 

Widmer had found several saa process variables’ to be significantly 
lated to successful adoption including "systematic planning, imple- © 
ntation, and evaluation of objectives; early and widespre}d network 

. building for support (especially among key decisionmakers,/opinion 

‘Teaders and gatekeepers) and extens dissemination." The follow-up 

study indicated that the projects, which remained, fully: tntact on a. local. 

financial footing three years later were tees that emphas ized, except 

: for evaluatton, all of these elements. F 4 


In light of such findings it is understandable that ‘the. State of 
Massachysetts: returned to MEC in.1976 to conduct a similar training’ and 
_ formative ‘evaluation of the innovation management ‘process. ‘This time, - 
however, -inputs were provided early enough in the project cycle to 
make a differencé two and three years hence when yonger term ictal 
and paintenance will become critical seSeas: 5 
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. The present, report provides’ a ives ot of these twenty-four projects 


-as they are getting underway; thus, it is more of a progress report than ‘ ge 
2. a definitive evaluation-of outcomes. : Extended interviews with each = ok hood dh 
‘ director form.the basis of the "Findings," and broadly confirm the, ~*~ ie 4 ‘| 


pattern which was observed and reported on by Havelock-MEG, in the pre-,., as 
vious year, The interview schedule covers a wide ‘range 0 /process a ae 
SSEUP Ss ile on: ‘ i, ne ci 
Se et ae aye 
@ The current status of the project, progress,» eg SES a a 
\deviations from plar, and problems yy po Ne a 
Mey gue Othe seTf-defined role of the project director | 


e Issues rotated to. the building of vale - ushtps , Be sa : 
with -decisionmakers and other intereste: and r 8 sae ae 
affected parties fs i aes oe 

® Strategies and activities enga ed in on the y & a ae Ac 

-» to meeting project objectives (such as diagn¥ Mr ee ee ee 

, and sensing needs, searching for new resourceéyof »- = 9» - “a 
aE _- information, materials, persons, and MONIES and Be os 2 eg 

* identifying alternatives ) as): , eter j : 


Ie : 
‘© Diffusion of information about the prdects ‘ - d 3 ea 6 
finding ways to improve the chances of project — er 
“ continuance~beyond the Federal funding pret A abe a : 


=" Project outcomes evaluation © -. % GOS BO i See 
am | " - 

Because these: interviews were eonduated: after the projects had been under 

way about six months, there was also an.opportunity to sense project - ae 

director reactions to four one-day inservice workshops provided by thé. Pig 
e. # — Department of Education and the Merrimack Education Center to build = 

ae management skills in the. aréa of innovation. process. ,; 


A major complaint’ emanating from these training sessions was that — 2 ee 
. inputs were too general and difficult to relate to what-directors per-. 4 
ceived as their radically diverse projects" needs and circumstances. a: - 
_ Partly for this reason, we have ‘chosen: to; ofganize this ‘year's report , Pd 
somewhat differently, Tooking more closely, at project content and ' 
/ analyzing process issues. in terms of content differences. To make this. ’ is 
Z. type of analysis possible we have broken the projects into four main: : 
"types" which we call: 
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shah each: category, obviousTy: there ane ‘vila “wide vardattons: $0" de ee ae 
fas , generalizations will stil}: be requit cross rather-heterogeneous sets, | 2% 
‘Nevertheless, certain. process - issues “Into” much sharper, focus with »  f., 
ar type of ,content Clustering. Pits, 4 ke a aed 
s 0 eager Tag, 
et ‘The veport: wilt have: seven sequential sections. as’ ‘eto: i bs ee Maat ‘ 
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: ” on Stal ét ae2 000°. itaveedi vided fpaeoctuie ays yields an. caveenae a ee 
a A tprodeets. ‘of. about "$7,000 “dollars, but funds a 


not ‘spread equally’ Pay am 
‘ Be : prajects.” “The: ‘wedian funding level was only 30;000: dollars.” a a ae 
an ue vty ye “reaind feaking a strong.skew to.theslower end of, the distribution; seven 8 
Rae: pee acts received 20,000 dollars or, less; however, eight ‘others .re-> ee . 
bate Aas tee fb 50% 0 sie gllars or more, suggesting.a kind of pita aaa h eat 
Pte Thee poe 500 ‘to 04 000 ‘dollars... nah. Sen cae 
arr ¥ ra Mtr gee paged ‘ a ae les OR at 
e ae ee me ‘Retosd: eet ‘projects pane fifty-eight percent of ‘the funds | ‘ere! allocated |§, .* % 
ee eg sd professional salaries,a figure which might:be significant later as ° a ee 
Me fa ai ' “vepresenting: an. estimate of the minimum apst of «long-term adoption for ° . 
wy the local: -schoot udget.*. Among the four. types, central service. projects . brane 
ae pe, ars cited For Seven Rpertent™ of the budget.“ on rarer salaries... : a Si gate 8 


i Y Ye im Among ‘the four project typess’ ncst: funds were: devoted to special reeds * Tr ae ; 
OM projects; ‘there weré «eight ‘projects in this.category totaling 351,000. » ; ae % 
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.. Strict cost-benefit terms, \may turn out to be superior, At this point 7 \ 
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: aoe - Le ie if ry ’ 
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” it is impossible to judge 
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» In this, -year' s projects there is.a ‘pronounced skew toward the upper end’ 


youth (grades 9-12), Juding most of the special need projects; two 

a combined high and middle; three concerned middle only (grades 6-8); four ¢ 
combined middle and elementary; one project concerned preschoolers and ° 
their parents and one, ncerned preschooters being tutored by 7th 

graders.’ -Three proj cts dealt with all grades in general. Ndne ‘concenned 


of the K-12 ear projects focused exclusively on high school 


. ; RY grades exc lust Vvelye.. : 
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The most ei statistic to deal with and yet the ‘most important in... 
a fundamental sense is thejnumber of students served. Cross-project 
comparisons can be invidious and at the - ‘same time ‘meaningless since one 
project can claim to serve |\thousands of students for a few pennies‘a head . 
whereas another appears to\ ‘lavish thousands of dollars on a handful of 
students. The latter ma: may have vastly ‘greater impacts and, in 


“this caveat in. mind, we can\\note that projects tend to be of two var ex oe 
ties: mass servers and special servers. Generally, the mass server of 
are in the central service type and their effects on students are © ~ 
expected to be indirect. The special. servers are the experience fusion 


‘wt .serve the public interest better, With 


| and the special needs projects. ‘Curricular infusion projects dre-of 


both types, but more generally are mass servers. Most special serving: 
sprojects also have. the poteritial, through dissemination and innovation, | ;. . 
. to become mass servers as well, and at Widmer is corrects their surviva} 
may depeng on Ths | tr fis 
General facts about the 1976 generation of Title 1V- e “projects fee’ m, 

y Massachusetts are summarized in Table 1 which follows. ; 


{ ha v4 


Widmer, Jeanne L. "Tnnovations and Bureaucracies: A Reexamination 
of Diffusion Strategies for State and Local Systems." A Paper’ presented 
to the American ie Research Association, Apri 1977. 
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| _ GROSS STATISTICS res au merousend He Heer Ny, 


| Gadi Students Served | 


| Experiente 
Fusion - 


Curriculum | / 182,000 | 26,000 
' Infusion 


Special 351,000} 43,875 
Needs - ‘ 
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HS 
HS:. 
‘HS. 
HS 
MS 
ES 
HS 
MS 
Pr 
S,K 


Services: : | 160,000 | cy 


yon _ 
TBE 


"882,000 


*Many prajects. serve mixtures oF levels. Therefore, totals are greater than 
number of projects. 


**Very rough estimates based on proposals and jateieds, 


werexcludity, Fitchburg's testing program which nominally affects a11sstudents even 
; though ae a io may be ReSeen to special SRT ’ 


Levels]. ahbealais sd Direct 
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curriculum focused on the restoration of abandoed pri 


THE expERT ENCE FUSION enthects 


From many points of view the 1976-77 IV-C projects of areateet: interest 
and perhaps greatest potential are those which link classroom: ‘learning . 
with life anc work experiences outside the school. Four ee fit 
clearly into’ this pattern. : 


A brief description of each of these expertence fusion projects follows: 


1. Somerville: "Project Outs ide-Ins ide" which makes use of 


vacant land adjoining school buildings as sites for student gardening , . 


projects; gardening activities in. turn are related to topics of health, 
nutrition, and environment in many aspects of ‘the curriculum.. Although 
operational only inthe sixth grade so far there are plans to involve . 
grades eight and ten and eventually to develop a K-12 curriculum. 
Thereis‘also a significant community*outreach via. cable TV and other 


. Media, encouraging ahanie in health and" nutritional haltits. i 


for the production of a vartety of agricultural goods intended fo 
‘market, The production-distrit ution chain is, traced from beginning to - 
end, “picorporating’ curricular elements of mathematics, science, art, 
graphics, journalism, government, economics, etc. With the cooperation 
of. faculty in all these areas a thoroughly interdisciplinary experi- 
ential learning environment is atic The project also anticipates 
earning income frost produce which will significantly reduce the costs 
of program adoption for the,school. There are also many community 

MUR eAen aspects. * af 


2, Turner Falis: "Project OP leds a makes use of a six if lot 


. 4 Wilbraham: "Project Blueberry" calls upon students to make 
comprehensive investigations of their suburban political-social-economic 
environment with the’object of producing TV tape\shows for local airing 
via a local public access channel (cable). Although. inspired by ‘and 
currently limited. to the classes of two English eachers, this program 
also has potential fusion effegts across curriculum areas and appears. ~ 
»to represent 4 meaningful and vivid experience for youth not only in 
learning: by doing but in making significant contact-with a social 
environment which has been heretofore remote from the classroom. 


4. Boston: "Project Shelter" a complete al ternative . high school 
ate; résidences 
in blighted urban areas. This project has all the curricula? fusion 
and community outreach possibilities of the other projects, and, like 
0.P.U.S., has the potential of producing outcomes of economic value. 
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‘As a group’ the projects are. admirable for their potential for turning 


on students, making many acadé@ic subjects come to life im ways which 

illustrate thejr real life and occupational relevance. They also have - 

the potential of building. understanding and goodwill between generations, 

between students and teachers, students and parents, students and public x 
officials, etc. The school becomes a much more visible force in 

conmunity life in a positive way. At the same time, all these projects ,- 


* are demanding in’ that they call upon diverse skills, collaboration on 
many levels within the school, and’ between the school: and the community, ~ 
‘and extensive planning to make sure that students, materials, curriculum 

‘ elements, and facilities converge. In conformance. to Murphy's Law 


(that if anything can go wrong, it will) it is not surprising that. _ 
these projects stumbie and that the project directors usually have their - 
hands full. furthermore, according to Pincus (1974) and confirmed: by 
Widmer (1977), radical changes which threaten the basic stru e of 
schools and schod}. systems are least likely to be adopted the long: 


i 
_Yrun. There is every reason to conclude that these are radical innovations 
because they violate the traditional separation of classroom and the 
‘outside environment; they violate the traditional structure of the 


traditional modeg of instruction and. management, of squdents within 

schogls. For all these reasons, it is especially interesting and ' 
jmportant to look in on these projects, in their earlier stages of ; z 
development to see how the process of innovation is being managed. 


curriculum and ef of instr departments: and nen, of sh and. they violate \ 


Our analysis for each type of project will follow a common framework : 


@ Descriptive data cs \ er ee * 
@ Director's role, ty Sony gee 
; . Wes tHe 
@ Relationships . at 8 ( i Re 
“s ‘ 
@ Probtem-solving strategy (including need sensing,’ " } 
acquiring resourcés, and, action alternatives ): 
.@ Dissemination activities o) «M p 
@ Continuation efforts 
@ Evaluation including self-evaluation procedure and yf ‘ A 
estimates of progress to date from tine point of view e 
of the project director and the interviewer } 
¥ 
= | 1 = \ 
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4 DESCRIPTIVE DATA, Be ig we, , | sag 


‘These four projects are similar in a “tie of ‘basic descriptors. Three 
- of the four are funded at about the 40,000 dollar level, the’ fourth, 

, Boston's Shelter, with, a complete curriculum to handle over thirty-six 
students, has 65,000. dollars. All but 0.P.U.S. allocate a’ ‘Tittle over 
half their budgets to professional salaries. All these projects are 
aimed at high school students within one building.and so far serve - , 


only a fractioh of the students in that building. All these préjects - . 


also require the collaboration’ ofthe project director. and'‘two or more ' 
teachers. With the exception of Piuetery” these’ teachers tone from 
different. departments. ‘ 


THE DIRECTOR'S ROLE Fae 4 boris 


From last year's ‘itevvians was Bouetane that there were four basic 
functions which were. typically performed by project directors, Some 
\ended to focus on one or another of. these function$; many assumed - 
more than one; and,a few "do-it-al4" directors appeared to do all and 
more. At least three of the four directors of these experience fusion 
projects are definitely in the "do-it-all" category: they are bundles: . 
of energy and-enthusiasm, some of'which appears to rub off, on others. 
Tis four eateaor tee break down as. follows: 2 


TABLE 2 


Racilitator ‘Communicator Guiding Lights Other | 


Now Later °Now Later Now Later 
ie % % % % a 


‘. Outside- 


Inside 
O.P.U.SS 


; Blueberry 


SHELTER ' 20 * | 30 S| 
; an . (teacher). |. 


Average. ““ 
Percent 


- Bene 


oe, 


we % ' 5 2 i 
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These data suggest that three of th¢ four see their role in somewilt 
similar terms. Only the Blueberry director sees his role primarily as” 
_that.of a manager. Only the Sheltér director had substantial aa rane} 
‘rote involvement--as’a teacher to/the students in his project... Other’: 
points worth noting in this tablg . .. ee 


@ There is almost equal. emphasis on the roles of! ee _ 4 
ienager TACIT br, and: guiding bight ' a jo 


» ‘8 _Someyhat lower eyphasis én communi gation oe ; be fe 


—@ There is littl? sense on the part of any ‘that 
‘their roles wll change substantially as! the? eal 
project goes Along cag 5 (a 


@ And to ‘the éxtent that tiolguls. any shi t. it ae . % vf 
is in the direction of increasing the commu= |, j be re. 
; nication function (for, 0.P.U.S. and Shelter) . 


In response to the question of other roles or tonetions! hestdel ‘these - 
fpur, directors ‘reported “teaming,” “orchestrating,” “looking for new 
sources of funding," "looking for ways to expand the program," eee 
“exploring sites for next year," and "teaching," although only tie toe 2 
eiaba ob gee was accorded a specific time percentage. All project 
director bia ob 100 percent. of their work time to the ‘project. ROR . mM 
most it &ppedred to be ‘an all-consuming activity Secupyme much more | me 
than the ‘conventional forty, hours. a Ae 8 
In describing their roles, two enphactaad that they "do, everything” , » 
and one ‘admitted that this led to trouble at times. The other pe ; \ \ 
emphasized their roles as "coordinator," "facilitator," and ey r 
"orchestrator." The most important, “attribute for performing theif %y an 
roles successfully were seen as "keeping many ithings going at once," cin eS 
“aggressive persistence," "patésenge," "getting along th people," and’ = «' ™ 
“dealing with people." Only one mentioned specific ‘techni ¢ fal skills. 
iy NY 
In summary, dtrectorsrd these’ experience fusion projects:'tend to be 
high energy "doers, "devoted to their work, and one could say their . 
“mission.” There.may be a possible danger that their projects are too 
closely assbciated with themselves as people and too dependent upon. 
them to survive and expand without these particular individuals involved. 
Blueberry is the exception to this pattern in some ways although the 
success Of Blueberry may depend on the continued energy investments of © 
‘its two _teacher-originators, 


"BYILOING AND MAINTAINING RELATIONSHIPS TO KEY PERSONS 


As noted above, two of the four emphasize people-relating elie in 
. describing ‘their own role. ‘It is clear, however, that all four projects 
- rely on a considerable amount of administrative support from: superin- 
tendents and assistant superintendents. In three of the four, these 


« 


y 


* proposal thus assuring a ‘considerable’ amount of institutional commitment. 


% ie 


onaa first name basis with éverybody; another noted that the schoo}, 
committeé president was an old friend; another noted 206k the superin- ")- 
tendent himself was a, farmer and hence took a special interest in the 
project; in fact, the assistant superintendent had written-the originat 


relationships were. Suthep personalized; one made ‘the point that he was “/ . 
4 


The Shelter Project is itself sheltered from Boston's sometimes stormy ew 
educational weather by a strongly «ipportive school headmaster and.the ” 
fact ‘that the host. institutton is designated as a nmagngt" school. 4 


All “four: projects Sgurtad ‘some kind of problems in, the. area of buttd- 

ing relationships. One director complained that he did not.have enough 1+ 
time to meet with teachers; another lost the participation of a teacher »« 
with key technical skills and good student rapport because his. depart- . 
mental colleagues pressured him to get out,, seeing the project as. a a: 
threat to their more traditional approach to providing such skills; es 
another had a key staff person.walk out. Three of the,projécts had* 
considerable logistical problems and delays due to blocking somewhere 

up the line; in one cas@ it was the school committee and its. chairman; 

‘in another case it was the business manager of the school system who 

delayed ordering necessary supplies. The Blueberry Project had early 
difficulties getting cooperation from local TV stations on whom it: 

would be dependent for project fulfillment. Thus, in each case probtems 


“tended ae unique to the type of project and involved a very: complex, : 


network of, peopleeinside and outside the schools.. * 


- From our point of view, however, what appeared most striking and 


deg across projects was the lack of planning or deliberate strategy 5 he 
“the part of these directors with regard to relationship building. = DP 2 
This would appear to be.a significant shortcoming for effort which depends... 


- on such a diverse set of connections within:the school and within the 


_ then, lots of media;" sey but of the task;' 


community. Project directors seem to have some faith in "doing what | a: ¢ 
comes naturally" as far as relationships are concerned and may have an |. 
inclination to depend too heavily on their own dedication” and energy 


and the goodness of the idea to Sanne them. eee difficulties wih es 
others. ‘ 

* et “ 
PROBLEM-SOLVING STRATEGY fi : z i ; 


We asked directors to describe: their own dsigatagy om model. of’ change" 

as exemplified ‘in the project, and as in the previous year the question. oe 

did not yield. much response; apparently project directors do not have an 2 e 

explicit strategy worked out.or articulated. Here are,some responses: « 

¥WChange strategy is all over the place, setting objectives and re 
"following the path 

ing on strengths--operating in a lol 


least. res istance--advocacy-bui hao, 
“these project directors spontaneously — ra 


key." In‘\their responses none 
mentioned the steps of a problem-solving approach.as: described in aso 
Havelock's Guide and elsewhere; i.e., feed sensing and diagnosis, search ° 4%, 
for resources, generating alternative solutions, and test of chosen ioe 
alternative. In response to specific interviewer probes on these topics x 
we found the Soe 
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ty! Biawneta’: 4 F : 
.  \ Three of the four, projects invested minimal effort in iebic a 
. diagnosis assessment after the project was funded. The Shelter ° ‘ 


’ director felt he had discussed needs on an on-going basis with the 36 
students. in the project and the Outside-Inside project had begun with 
inquiries addressed to’50-60 teacherS, The 0.P.U.S. director cited an 
HEW report which: had documented the need for thts: type of eee V4 


2, “Financial Resources . ap ds . oi y 
‘Three of the four project direst’ expressed. dissatisfaction: 


with the amount of funds available for the project (in contrast to 1976 - 


when almost no dissatisfaction was. expressed from over thirty’ projects) « - 
In the fourth (Shelter) fed tape blocked the use of funds for. months, _° va! 
forcing the project director to make an unreimbursable, voluntary 
contribution sea his own savings of 1,000 dollars. Three of the four ; 
projects also had significant additional sources of support beyond ~ & 
IVsC:; .One had a 5,000 dollar grant ‘in hand.from a private foundation ' 
which allowed circumvention of the red tape problems besetting. other _ ‘ 
projects for purchase of matertals and equipment; another made good use ~ — @ 
of ESEA mini-grants to teachers; "another had prospects \of receiving a 
3-4,000 dollar grant from the Massachusetts Council’ of the Arts and. — 
Humanities. In general, it could be said that these directors were 
fairly resourceful in ferreting out boiaida aia sources of: financial 
support. ; 
“Informational Resources bathe ae 

_ ‘Three of the. four projects had: shedtaaia roots and forerunners 
in other, districts inside and outside Massachusetts. Transfer and ’ 
adoption of “these programs was, therefore, presumably assisted by the 
National D1 ffusion Network although neither the NDN nor its state 
facilitator gets any mention. There is also no deliberate use of” ERIC, 
and minimal use of literature sources or expert consultants from univer= 
sities or elsewhere. Resource search is thus seen almost entirely as a 
local matter, a fact which may partly explain the negative.or indifferen 
response of directors to the MEC workshops, (See below -for more dis- 


cussion of this point.) \ | i 4 . 
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3; Generating Alternatives for Action 2 


Two of the four responded simply "no" to the question, "Have 
you considered or developed alternative solutions for project objectives?" 
The Shelter Project was forced to, go temporarily back to a classroom 
farmat for the winter because of the weather and no windows. Thé 
Blueberry Project discovered a new additional focus for project activi- aes 
ties surrounding the Hamden bi-centennial, but otherwise plans essentially : 
to repeat. the first year's Soe | in years two and three. 


and involvement activities, in two cases’ involving Tocal cable TV. 

_the projects have visible output which should serve to give them a 

positive public image which, in turn, should be useful in promoting: a’ 

positive image for the project and for educational innovation in general. 

' This potential has not been attended to and it may be too’early to do 
so. As the projects. advance into. their second year, however, it.will - * ri x 

_ ‘be important. for directors to raise their sights. significantly if ‘the eee 
full potential of these projects for reform of education within the P 

state are to be realized, In the case of Blueberry there could be’an ° © 

: expats ion, into other areas of the curriculum besides English with the ~ 

~* {nvolvement. of many more teachers; the approach. also seems applicab] 
‘to: other communities suggesting ‘that it would rent ‘the eae of con- 

ferences or state-wide inservice waoicaate 


| ga : : "8 os pteeee 
DISSEMINATION AND DIFFUSION : ix tes a ’ 
As-noted earlier, al these projects have. significant communi ty a ll le 


Eon 


There could dis be significant 1 ngs fo Blueberry from projects rs if 

0.P.U.S.. and Shelter which combine many aeRO areas and fuse them — ; 

around the task; there could alg be learning for-the latter two from ee \ 

both Blueberry and Outside-Inside whieh make, creative use of TV and 

community interviews and quéstionnaires. to build awareness, interest, M 

‘ and support for the’program, As in other ‘areas, there is so far a lack és 

of articulation.of strategy or specific’ ‘planning regarding diffusion ; iar 4 
either inside or outside the ee \ a 


CONTINUANCE EFFORTS 


ne a eee 


a a 
se 
As'noted earlier, sécording to the Audinae of the Widmer Report’ 1977), an: 
- tnnovative project often has rough gaping when, it:Comes to continuation °° 
because of: the major changes which it forbodes a the structure and 
methods of education. In spite of this ’problem there has not been’ Pal? 
too much attention’ paid by, project staff tovbuilding ‘in the kinds ef. 
. institutional mechanisms and guarantees which are required for a 

term, Outside-Inside i§ so far as aa add-on with no commi 
for local funding although there is suppose@ to be one full-time 

equivalent built in by the fourth year (the project director says ' 
significantly that 4 Will not be him because he. will want to go on to. 5 
Other. things by then). Shelter has a pledge from the headmaster- that 
_, the project will be fully absorbed in three years. Again, the director. 
says he will not stay in any case. As for Blueberry, there is so far a e 
community which "seems supportive" but this support has..yet to be turned 
into, commitment.’ Finally, continuation plans for 0.P.U.S, seem to hinge, 
upon the sale of. produce ‘for a profit. which returns to the project. 
While this notion seems salutary it also seems Tike a tenuous, hope for 
the yield from a five-acre lot;.self-sufficiency appears to be & grossly 
‘unreasonable survival criterion for a project which has so many ‘intrinsic 
educational and an benefits. 
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_ _« None of these, ‘projects. ‘has’ lindergane an adequate évaluation: 
Three depend upon more-or-less subjective ratings. by byoclenss journals, 
logs, and judgments by advisory:councils.’ On the other hand, all have 
- the distinct advantage of having visible products which could be’ rated | 
. and judged by insiders and outsiders on.more-or-tess objective.criteria, 
It was a matter of particular concern that the secondary benefits (and 
Hp aw) «COStSy Tf any) of these projects get'a thorough review since it’ would : 
a - appear that directors are only dimly aware, of these even though such - ie ay 
4 ‘information could have’ great persuasive power with community, school : ak 
; "boards, and administration.» Awong these secondary benefits appear to be 
“ , ‘much higher student morale and interest in sthool and TnGraREEs my be 
. wu8: will: for education and. for youth, 


mt 


Results td Date Self- shenortes by nineties ae in 1977 


Speed: Only one project, "Blueberry: reparted Ytsale’? ‘to OPS S ae 

on weisdutes the others expressing yarying degrees of dissatisfaction . : ial 
with. the slowness of progress to date. Weather was a hampering factor, 
severely in the case .of Shelter. Red tape:also playeda nei eens one 

; director admitted to being ‘too ambitious." me eG ; 
Me 4 ( . 
: Ds Number Affected: . Two projects were serving the’ number a 

students proposed while the other. two were serving somewhat. fewer in , 
direct terms. Numbers ,become, especially important important, when. per- 


‘4 student costs are high as in:bothyShelter and Blueberry. Both these ae 
projects have very great expansiog-diffusion possibilities within the *. 
school itself. WIth, if exploited, could raise the cost-benefit ratio et bi 
-considerably. Pi . \ 


ee Amount of Impact: All directors sce pleaeed with thes 


ho oN te amaunt. off ee on the students who are so far inVolved in their 
2 projects. The,Shelter Project has turned out’to be an excellent vehicle 
for improying nter-race and Intersex helations asvwelt as helping to ie 
retain pa ntia) dropouts. a Yo ot 


* When asked to’ point out the major items of criticism: they 
. would. level at their own projects, two sighted slowness, two cited 
‘ insufficient ties. so far i ab ae one olted tack of time 
, Spent directly. with teachers ‘ 


Reflections of Interviewers and External Evaluators ° 


w In our Judobents all these innovations have extremely high 
potential as changes which wi11 bring multiple benefits. to school and . 
community at modest financial cost. ‘They can change attitudes and make ‘ 
the curriculum more relevant. They support creative. and relevant 
Reaching in methods and content. We feel, however, that all these . ‘ 
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ee pindarts are in sprious danger of falling very ie short of nis: : 
_ Potential fora variety of reasons, vizi ute 


uk @ Most of the project. directors are not stressing : be 
m4 the wide educational change potential of their . 


projects 


The projects-appear to be devandant upon the 


energy, enthusiasm, and creativity of the director 
even though the aarARsOr, sees. nis dl as ny a. 
temporary one 


. The lack of eine. ‘awareness by ‘the sitrector of 


his role as a change agent: arid ‘the requisite skills 
surrounding such a role 


The Tack of : specific planning and execution of a 
strategy of building relationships with ‘those 
persons and groups whose long-term support is 


Necessary for the survival of the. project 


, The lack of articulation of a strategy of problem- 


solving which is sensitive both to ongoing and 
changing needs of students and teachers and to 
the evolving information resource base outside 
the district ¥ 


The lack of articulattdh of: a. strategy of communi-~ 
cation, expansion, and diffusion which takes into 
account both the full potential of the projects as 
image! builders, and the knowledge which is, available 
concerning the’ diffusion of innovations in general 


The lack\of a consultative and inservice function 
for IV-C directors which adequately motivates them 
to percefve and to remedy the abovesT (eked 
deficiencies 
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THE, CURRICULUM INFUSION PROJECTS : , ee ae. ae 
s* ; + 

“ Seven of this year's projects involve various ‘kinds of: currfatah\ ae aie 
changes, or changes which affect students only tn certain, subjects. Four a fan 

of these concern the arts, one history one English, and.one nutrition.» han 


A brief description of each of these curricular ‘infusion podects ier ea 
follows: ae ee ee 
: 1. Greenfield: "Children, irttsts, Sound, ‘dilors Cameras" -is a. PBs sag 
"collaborative effort between the Arts Council of Franklin County and “ ' 
the Franklin County Elementary Schools" to provide liaison to arts. 
professionals, provide materials in kit form, and establish an arts” 
resource center. In actuality, most of the effort, is now going fnt@ + 
the preparation of kits based on kit development in a prior project in y 
Hanover, New Hampshire. The modest Title IV-C grant of 11,000 dollars S ‘ 
supplements. a, larger amount (20,000 dollars) Sener ibutes by. the Arts '. 
Council itself. 


2. Boston: "Music in the headatitie, Curriculum" invoTved workshops oS 

for forty middle school teachers to prepare materials and activities , 
that wil "better prepare students" for attendance at youth concerts . 
in BostoWw's Symphony Hall. Such activities are supposed to "bridge the - | é 
gap between the arts and regular classroom subjects such as social a 
‘studies and language arts." Budgeted at 57,836 dollar’, it is the 
‘ largest project of this type funded this year. ‘A relatively smal). . 48 

proportion of these funds goes to supplement salaries on the regular 

school payroll (16, 500 dollars). : ‘ 


3, bith Attleboro: "Project Lite": seeks to. «Ines arts-related 
content in various parts of the curriculum througlt teacher-developed od 
materials, The ifitiating vehicle is a summer workshop for twenty-three 
teachers, in’ which. existing curriculas for. he _ third, sixth, eighth, and ~ «~~ \ 
— tenth grades are identified and revised sson plani changes and in- ‘ 
school inservice will then be conducted: of esson teachers throughout 7 
the year. The budget is 30,000 dollars with near y 20,000 eats. for | ar 
professional salaries. : 


, * Lincoln: stan Through Art" project uses the facilities 

and resouréee of the DeCordova Museum to infuse visual art content 

into the "existing Social studies curricula in elementary schools in 

five towns: Lincoln, Carlisle, Concord, Sudbury, afd Weston." The 

key to this program is the.use’ of the Museum's "docents" as classroom 

visitors and assistants. to the regular teaching staff. ‘They prepare i. 
materials and lessons; after one or ‘two years, regular teachers can- a 4 : 
assume the same activities without outside assistance. Budgeted at 

30,377 dollars, the groject makes extensive use of volunteer help and 

contributed services by the Museum. { 


¢ 


r} 
’ In sharp ‘contrast to” ‘the’ Secuprane’ fusion swietanss? ‘host of these’ 


-. over several other activities and hence devotes much “less than hal: 
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‘mentary cucriculum (especially grades four, five, and six) in social Le 


wr a . ea al . r 
5. Springfield: "Colonial History peace attempts to Yhuse- 
an understanding of Tocal history (natural and.socia¥) into the“ele- 


_ studies and science. The principle vehicTe is, the development, of os git 
““suitcase exhibits.” Teachers participate in this development effort 4 
through a mini-grant program.. \A six-week summer workshop for fives 
teachers and a cogrdinator launched hie prognanie he tia oas 

, 28,600 dollars. 3 


aie Springfield: Has also beet asatied a dat ae of a 350° nga 
dollars: for a tenth grade.:"English.Composition Laboratory," involving 
three teachers and'a university-based consultant. Much of the funding 
goes for release time to these. teachers to. conduct one laboratory class‘, .. 
and to, develop a. modified curricl Tum to teach compos tion SkITIg,, a 


"2" Southwtcke Hae: received . modest grant of 13, 400 Yollars ‘Fors | 
the "Integration of Nutrition Education: in Grades. K. throu h-12,% So t= 
far the project has concentrated on the middle school grade levels; 

. but, employs a variety df.strategies at various levels, essentially, to’, 
heighten awareness, The project appears to be tomptex ,and ambi tious | 
BSEEI Suey in aia of the amet budget. ig” nokae *s - = 


; ae ee : . a 
DESCRIPTIVE DATA 7 a ey: ) : - 
As noted. in Table 1, relatively j Vittle money wad sp Hof this: type of 
project. in both relative .and absolute terms, pet e budget’ per proje \ 
was only 26,000 dollars witha range of 11,000 dollars .to 58,000 weet, ee ee 
_(Bos ton- Music). “All of these. projects. put some’ emphasis on, the develops eS 


» ment of materials or packages which could affect thevexperience. of! large... W to Ey 
numbers of students directly and could potentially havea useful life.” . 


beyond the termination of funding. y Hence, the estimated number of —"’, es 
students who ned tT affected dirdctly was quite high, over ;3,000 per: Gis are 
 project-compared Jess than 140 for either sail ty fusion om aie a 
bag. eas prosecte, a Se al. ws ot fig 
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THE DIRECTOR! S ROLE re ys eh es: ; wk, Ki 
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j at wed 


projects are headed by a person, who had administrative responsibi1 ty 
time - 

to the project. The two exceptions are Greenfield and Southwick where 

both directors complain of overload and underfunding. In:both these - Gi a, 

cases the director claims to function»in. all ici mutes ane is te BY 

heavily engaged, in. promotional efforts. ; eo a 


Of the four role types suggested, the most popular was’ "guiding light" gf 2 
peers 36% of time) followed by "facilitator" (23.5%), "manager" «  ° bike ow? 
)(20%), and "communicator" (17.5%). One director indicated that 15% of ye ae 

his time went to aevasldali on oropSa ee: The most important. tribute 
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these project directors ih AUSSREOM 


Because so any, of "thers projects involve iesteriats development, ‘they! uw 
‘may not be’ as visible.or as vulnerable to key power figures in,the > |. 
school. adminTstration or the community. on case, of the rel a iif Pe 


few difficulties reported, apathy and ins 


ones. ' One director felt that educators in general’had an unreceptive » - 
attitude toward the arts; in this case thé direotor was from outside }. 
the school system and had' relatively weak ‘links with key administrators , Ms 
‘at.ahy level. This director expressed a desire to develop parent 
.* , pressure groups to force the schools. to change. In contrast, anather. 
project director’ emphasized. the need i keep: cool and ba diplomatic. 


Among key persons named there, are no dubavteae’. The Super aust dis ’. 
cited most ften, followed by principals, teachers, and parents. In — 
six of the seven, teachers~participate quite: actively: in materials ~* 
development and hence can gain some sense of “ownership” of the project . 
and its outcomes’ .Tellingly,. the one project without much: teacher 
‘participation complains the most about* mal aplonshp ALR eae les 


- On the More technical side, scheduling difficulties’ were reported. in — 
two instances, one. the result of attempting to coordinate workshopsefor 
joint participation of. different school districts, the other, (toa 
complicated by the terms, of the teachers' contract, - a 


PROB|_EM-SOLVING STRATEGY 
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n two. instances and “idachniteal <sk{11$", | As 
in-andther. | €vidently not: too guch thought ‘had | heen given by any of Bx oti 
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ivity are the principals 


Directors. gave very littte” eeblaataiin of their fasnucttve strategies : 
for developing, conducting, or implementing their projects. Materjals 
development, either kits or lesson plans or curricula, or exhibits) .. , 
_. formed the core strategy, supplemented by site visits. Also important. 
- .. was the use of existing resources -in the community such .as museums, ‘ 
_, Symphonies, and arts councils. As noted earlier, teacher participation 
‘was also a part of the strategy in most caseés,, Thus, all the elements 
- Of a good strategy oo present, although not anneso tay or con= 


y sidered as.a "strategy" | s3) ja 


‘s 


; " 4! _ specific’ process issues we found the following: . 
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‘| ¢ Loa ‘Is Diagnosi} | 


# y 'y 


as 


Three projects claimed to Fave expended a reasonable amount of 
effort on assessment and diagnosis of needs but in no case was this : 
"systematic and in general we would conclude. tae this is a aI 


"ar all these projects. 
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, 2. Financial Resources ae . : 

is Two #Pojects expressed disatisfaction with “existing funding” 

: levels and a third was clearly hampered in this respect. Three had , 
substantial income from other sources and one. operated with considerable " 
in-kind support from a community resource (museum). However, Jittle', 

. entrepreneurship was exhibited by any project to enlist additional y = -.. ». 
financial support. | | a * a. 
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Informational ‘Resources i Was Y, 
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“Most projects did not exhibit any aggressive outreach effort — 


for informational resources. As we've mentioned, the Greenfield project 


: general relevance and utility throughout Massachusetts and elsewhere, . 


— 


made. some effort to adapt kits from a prior Hanover, New Hampshire’ ' 
project and other sources but found’ that essentially they had to Start ° - ‘ 
over.. “Similarly, the Springfield Colonial History project started from 
scratch, building on a prior bicentennial project.- While drawing #deas - : 
from others across the country, poey a Boston museum, The Spring~ 

field composition laboratory relied heavily on the consulting services: : 
of a professor from a nearby university,. Finally, the ‘Southwick 

nutrition prosect made heavy use of material, prayided by the Dairy . hs if 
Council, CEDIS, the Minnesota Special &ducation CurrBculuh, and other 

print sources. This would be the only pevieet to date which could be " 

said to have had a comprehensive sear@ activity. , aa. 


ef te, 
3. Generating Alternatives for Action. 


“After the beginning of the projects, only one considered an 
alternative approach, the Greenfield Arts, which chose to develop their. 
own’ kits rather than relying on Hanover's kits as originally planned, 


DISSEMINATION -- DIFFUSION 


Five of the seven projects had or 4 ed newsletters; two mentioned 

newspaper coverage, and one planned\an extensive serfes of workshops for  — ‘ 
promotional purposes. Neither TV nor radio coverage was mentioned for’ | ~ 

any. Generally, there was no coherent planning for dissemination, 

This Seems especially surprising considering the nature of the projects 

and their focus on materials development. There was little indication 

that, any thought had been given to dissemination beyond the original, 

target group, although in most cases, materials would primarily have 


A "4 .: ‘ , ; ie 
CONTINUANCE, EFFORTS mf j 


In. one case (North Attleboro) there was evidence tof wholehearted support. 

by the school board and hence an assurance of future funding; infanother _° AS 
case the project was seen as having political leverage and hence has 

been el of other Federal grant support and has been written » > 

into sOme school-budgets. For the.othefs,.the future seems rather 
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o~ aussitanais, Most have no lang ‘Mn this regard and few seemed: to. ‘hav 
given the issae, much thought as of February 1977.' Although’ such. projé ec tg. 
¢ . .genérally provide little threat to existing interests. and fils 
os their add-on nature makes them obvious targets of sia aa a hen” e. : 
‘ schools are scrimping. PP og — Ge er 
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- 1» Mery little was reported in the interviews on evaluation: pro4 are 
cedures. It would appear that most evaluations are planned to be et 
minimal. Considering the fact: that in almost every case a concrete. . A ee 
product is ptanned, it would appear: that quankjtative evaluations of PF at 

. reasonable vigor could be designed, * ” 
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Results to Date Self-Re orted b Directors 


eed: One project, expresses aiertrunteaiond ‘but there” ns 

is general "tat sfactions : F af 
Nb. Number Affected: No indication either of cuts. or increases 

in size or scope of target audience, v a 


Ce Amount of Impact: ‘One project director* thought that the of fi. 

} : original approach of using an itinerant teacher-consultant might be too 

\ fragmented. Another was frustrated by an inability,to raise the level\ » ‘ 
of awereness and. concern by sixth’ grate teachers in nutritional issues) x's ene 
while a third expressed general frustration at the attitude of “educators” f 
toward the arts and change in oe rection, Otherwise, there was « ; 
general satisfaction thatyth ects would have, the kinds: of pytaones ‘ 
and aE that. had bee ticipated, i 


" Reflections of Interyiewers and External Evatuators } 8 ye 


_ Most of ‘these projects are following a rather well-trodden 
‘path to,educational reform: participation by teachers in materials hist 
development with rather modest and localized objectives. In at lé@ast 
two cases, they also appear to be.underfunded and in at least thre 
cases, staffed by persons with insuffi@ient experience in project: 
management. it is especially surprising that no more thought has been 
given to how the materials, once developed, can be continued in use and + 
spread to other users, even within the oytginal jurisdiction. Within 
serious attention to diffusion concerns before thes€ projects go much 
further, they are unlikely to realize their on-paper advantage over 
"porta projects in the numbers of students directly served. ‘ 
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PROJECTS FOR YOUTH WITH SPEC AL NEEDS’. « j 
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2 VA . ; “ 
Eight projects respond to the special: needs of various school or pre~ ° 
school populations with émphasis: on the poténtiad school dropout. ; 
Five projects, specifically, address this group, attempting to relate 
__' to potential dropouts at+grade Jevels.nine, ten, | eleven, and twelve, ° 
» One provides parental guidance for "poor risk" preschoolers. One pro~ 
JA vides cross-age tutoring betwéeh white seventh graders and day-care x 
preschoolers. And one provides awareness-of career options:in technology 
ee, for women, Clearly, therefore, these projects:are widely divergent. and 


? 


deal with various elements of the academic spectrum. 
' Brief descriptions of each follow: 


, 1. Amherst: "Regional Alternative Learning Environment" pro- 
., ides ‘a highly individualized program for "lost) students" based on a 
“4 diagnosis of their needs, and offers an interdepartmental team with 
one teacher each from English, social studies, mathematics, and science. 
Re-entry into regular programs is expected after one year. Grade 
levels ten and’ eleven. ‘ Para i 
“ i) ie 
* 2, Billerica: "ProjectswREACT" proposes a program for one hundred 
‘i high-risk potential dropouts which includes individualized instruction, 
: life skill classes, career awareness experiences, and counseling. iP 
In another aspect of the program ‘twelve students with "potential Teadept 
ship and helping skills" will be trained as "peer counselors." al a 


.3. Reading: "Project FOCUS" is a complete alternative high 
school program within the high school: for students "who are having *’ 
difficulty functioning full time tn the mainstream of. classes." Emphasis 
is on skill building in specific areas of English and mathematics in 
addition to "family group" meetings and an outdoor physical education. 

. program: emphasizing individual skill building and group)cooperation.. 
4 Behavioral contracting is asignificant aspect of the over-all approach, 
Re-entry to the mainstream is to be achieved whenever possible. 


ul 


~4, Shrewsbury: "Project SHAPE" provides intensive personal 
counseling, aS well as tutoring in basic skills. in small groups, to 
ninety ninth graders identified as potential dropouts, p¢ 
ry . , ae 


= 5., Fitchburg: "Project.Competency" concentrates in its first 
year on system-wide diagnostic testing to identify potential dropouts. 
It will identify survival skills for such students’ and develop a 
curriculum around: the notion of functional literacy. ‘ . 


6. The SPOKE Collaborative's project "Impact" provides workshops 
for parents and\"surrogate parents" for ‘preschoolers who have been ” 
o « be screened in kindergarten as "poor risks" in cognitive, affective, and 

Po ‘bila areas. Towns served are Easton, Foxborough, Mansfield, and 
i orton. | heer aN 


r gorized separately. 
‘ 7 
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; “7... Dorchester:- "WALK TALL" is ‘a cross-age tutoring project 
which brings white seventh 
preschoolers in a day care center. Basic skills. are emphasized. 


vu " J o 
8. MIT: MIT-pased "Project WITS," sponsored through the 


Cambridge Public Schools provides workshops and site visits for teachers © 


and counselors to heighten their awareness of career opportunities for 
women in technology. and science,/” This was ithe most difficult project to 
classify and perhaps deserves its own category. ' 


Indeed, the last three named projects are only loosely connected. to the 
others through the concept of "special needs" and é@ach could be\cate- | 
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DESCRIPTIVE DATA ; Move ee aed ; 


Again referring to Table 1, we note that about 40% of this year's 
Title IV-C expenditures were’in this category and that the average - - 
project budget ofinearly 44,000 was second only to the experience 
fusion projects. As in’those projects, there is 4 heavy emphasis on 
high school level; with the exception of the Fitchburg project which’ 
tests all students, the numbers affected directly jare relatively 
smal}, (136 students average for the other,seven projects). 


THE DIRECTOR'S ROLE: . : « 


“These directors, unlike many of ithe Sineteahed antictan atiechars: are 


invested full-time or nearly full-time jin their projects. Rather than 
being administrators, these directors are typically quite involved in 
their projects as team leaders, trainers, teachers, resource persons, © 
and doordinators. Among the four possible role,models, the most 

popular was "manager".although thase who stressed this role saw it 
fading in the second year of the project. "Facilitator" and "communica- 
tor" roles were each rated at about 20% of the total effort whereas 
"guiding light'’was a role which most of these directors spurned. 


.. Was seen rather divergently by different respondents as_"building 


relationships with subsystems and other service agencies" “communica- 
tion and. group skills," "mapping out the total system picture and 
seeing where you fit," "empathy toward youth," and "enthusiasm." © 

In addition Pate four role types suggested, some directors said their 
work involved "planning," "front end evading," "trouble sHboting and 
acting as a buffer," "teaching," and “working directly with students," 
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The single personal attribute of greatest importance in, task ieee 
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graders together with Black and other minority ‘ 
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suo MAINTAINING RELATIONSHIPS: / $e = 


As for other types of projectss superi ‘candanté, principals, ‘nd schooh, 


_ committees were al] Seen (about,.equally) as theskey relationships) to | \ 
~uild and maintain. For this type of aT dict, the: actual’ configuration 


may have been quite complicated in Many cases. One director found’ 


~ himself .caught in a conflict between a principal and a pupil personnel. | 


services director. In-the case of the MIT woinen in’ technology program, 


~ ivelationships to. department chairmen’ turned out to be the key in i 


“+ cation w 


ih stra gic questions were addressed as follows: 
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eababiisiningyered PT NIE, and Belong entry to high schools. be ere 


In terms of eatnique,- t ere were sep few surprises. Advisory com- 
mjttees tt Iped, and stress /wasjplaced.on regular. meetings and: communi- 
th top persons} seepy g these as informal as. possible. 
pnly. three projects reported any difficulties ‘in relitionsiips. In one 
ase, the principal operated in. autocratic fashion,insisting on detailed 
scrutiny and approval of the smallest bute -items, In another case®, 
some guidance counselors disputed the use of teachers as group gounselors 
for dropouts perfaps feeling that their own ral We as usurped 
or by-passed. / 5 
\ ‘3 
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PROBLEM-SOLVING STRATEGY, . \ ae 
There was also rather Atlerde barsenttons of problem-solving strategies. 
Two: directors say this in terms of rglating to students,/having a a 
"bottom-up development” or building rapport with students on a one-to= 
one basis, communicating. expectations to students of responsibility. 
One project saw its strategy as one of "infusion" of new skills,— 
responsibilities, and role’ images into the teachers' daily work. At 
least three projects saw the strategy as one of communicating effectively 
with practitioners and other key persons. For the MIT-WITS project, it 
was a matter of "cultural exchange" or "the intagration of people in 
different worlds." The Fitchburg director, thou lding a line 
administrative post, saw himSelf continuing to act\as\he used to act ° 
when advising the, system.from outside, a consultan eliciting change: é on 
a voluntary basig, constantly keeping people informed and in tune with 
the ep by atep progress of SGEIVESIES _* 


Diagnosis 


Most projects teported a rather high level of effort, sanity. 
cided on student testing ‘and screenihg rather than on system-wide © ° 
needs or. project needs. In \two cases diagnosis was claimed to be 
"on-going'/ or et a Mn at least three cases,, it Was made in 
cuit ve as well as qua ere eae 
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4 ereasonable amount of flexibility in project implementation. 


4 


2, ° Financial. Resources sr 


. Three projects expressed dissatisfaction with the current 
‘1evel of financial. support; two others reported sufficient funds, 

although one felt. that good use could be made of additional funds to 
expand the program; another project felt they would have some funds’ « 
left oe uy director noted that spending in early months of the ° 
project ‘lagged while credibility and staff capability were being built. 
She feared that funds would later be insufficient as things really: got 
rolling. . yy 


{ ’ r ‘ f 7 
aitictional Resources he \ ; 
r Several es in. this special heeds category shawed ini- 

tiative and crdativity in reaching out for informational resources. 
Two. projects were partial adaptations of NDN-validated programs from 
other states. Yet, even in these cases, "adopters" took elements front , 
various sources to develop their own programs. The Shrewsbury Shape 
project, for example, made some effort to adapt the Title III "New- 
Model-Me" program but found’ that it did not quite fit. their local 
needs. In addition, they conducted an ERIC search and: made extensive 
use of work by Casefield. 

Dorchester's WALK TALM staged relied on several sources in 

developing its approach to cross-age tutoring including the University 
of Michigan (Lippitt) eee the book, aman aneengn Teachings 


and the NYC "Youth Tutoring Youth" p 


Generating Alternatives for A 


‘ Five projects indicated some shift of course since funding 
» began, For Amherst, it was a more effective and-efficient’ method for 
identifying students in need. of the program. In Fitchburg, there was 


3 


shift toward more attention to individual teacher needs. In WALK TALL, 


the students were brought into greater involvement with the curriculum, 
and in Shrewsbury,’ there was a move toward developinent of a unique 
tailored program from various resources, rather than merely the wholesale 
adoption of one (New-Model-Me), Finally, in the MIT-WITS, there was a < 
shift toward more school-by-school consultation activittes. All these 
changes were relatively minor operational shifts ‘but do suggest a 
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"DISSEMINATION AND DIFFUSION . ; . a = 


.. 


“iva projects pepaile some kind~of discobination activity, usually 

within the original jurisdiction. Three others report no activity 

whatseever. Two projects report having their own newsletter (Billerica 

/and MIT); one other uses the collaborative's newsletter (SPOKE).. 

_/ Billerica also makes use of advisory ‘council meetings’, a parents 
‘advisory council, a slide-tape presentation, and a yearbook (being 

planned at the tim 


e of the interview), thus coming close to having a 
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comprehensive dissemination strategy.. In Fitchburg, there has also’ 

been a major’ Jocal dissemination effort including: slide tapes; a 
tion of scripts and transparencies, newspaper articles, appearances 

local groups such as Rotary, parent meetings, the formation of an } 
advisory council, and radio appearances. These activities ‘in Fitchburg | 
represent a rather complete groundbreaking for. what might be a series of © 


“welated reform moves \in that schaol district emanating from this. project. 


Finatly, the MIT predect has planned some. hiss gulls and regional dis- 
‘semination through a newsletter, appearances a profe$sional meetings 
and colloquia, and through the publicity channels as the National 


Alliance of Business, 0 fg’ 
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‘on board until February.» Another had trouble recruiting staff because 


SovFtwuance EFFORTS a pe 


..Four of these projects have dees some ‘sort of planning’ for continuation oe 
beyond thg,IV-C funding cycle whereas ‘the other four have not. | Of \ 
‘these, thé inost solid appears to be Fitchburg where fhe school system 

is fully committed to take over the project and hasyalready planned in 
this direction in the 1977-78 budget, with growth *to follow in subse- 
quent years. . The Amherst Alternative Learning Environment Project also. 
seems to stand a strong chance of continuance since 190% of the teach- ° 
ing staff is. already on local funds; careful documentation and a report . 
on the proje€t are expected to @trengthen the likelihood of survival, 

In Billerica, there is a phased program for school district pick-up of 

" project counselor salaries at the rate of one yer year; teacher salgries 
are already covered by contract. A gradual growth of local cost shding 
is planned in Shrewsbury, starting with 20% planned for next year. For 
some-of the other projects it may simply be too éarly to come forth with 
such plans, considering the need to build credibility for the Rai: the 
project, and the staffz . 


EVALUATION , os det ' 
1. ‘The Procedure |. o hes 


Most of these projects are not geared up\for extensive, 
rigorous, or quantitative assessment of outcomes at this time, especially © 
as concerns student outco Two projects considered student attendance 
as a significant measure o "success and ‘another indicated that they — 


_ wanted to get out of the testing business and shed that sort of image. 


Evaluation seems to be a rather weak area for this set of projects. 
‘ 


Results to Date Self-Re ported b Jalen : a or 
a. Speed: At least four ‘avadeuts indicate dissatisfaction ¢ 
with delays in getting started for different reasons. In-one case, 
the project director was not hired ‘until December with key staff not 


of the minimal’ financial and other incentives that appeared to be in 
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ee effect. for another, it was a matter of scheduling appropriate times , : 
' for, ‘meetings with ana? who had been. selected. 


be Numbers ‘Affected: ‘In general, peak waters involved 
seemed. to be close to those projected; in one case, \there\was concern \ 
that students were being enrolled in the program merely because: they’. 
were seen as trouble makers and not because they really belonged in the 
.“program, “For two ‘programs, Students were. ‘reluctant to participate 
\ because of; the refusal of the administration to grant full credit for 
the time, work, or activities involved in the innovative program. 


+ 2 + 
C, Amount of Impact: There seemed to be general:satisfaction 
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_ that if logistical, credit, and scheduling problems could be solved fh as 
| programs Would be’a great’ success and have at least as much impact as’ © ‘ ( 
mrsactaes | There was no ‘exception to this-view although one director . 
admitted that she might have been a bit too ambitious in her expectations, / 


Reflections of Interviewers and External Evaluators 
In terms of survival. potential .and in terns of promise of impact, 
this set of projects stands midway between the experience fusion and 
the curricular infusion projects. Because they treat special groups ~ 
with special needs, they are a bit off to the side and, therefore, less 
visible and less threatening to-the status quo than the former. In 
addition they relieve the traditional system of\some of its worst 
student headaches. Philosophically, structurally, and instructionally, 
many of these projects represent a break from school tradition and may 
es seen. by some as* the ‘opening wedge for a radically different approach 
in one nat tn general. This potential conflict comes to the surface 
one tance with the strong need of a traditional principal .to keep 
an ntact s of the new project under his thumb and in two other instances ~ 
in “the refusal] or reluctance to grant full academic credit for project 
‘activities, All this emphasizes the need for good public: relations with 
the school authorities as well as with the community.. We appear to have 
a shining example in the case of Fitchburg where. the project director, 
a part of the administration himself, has orchestrated an elaborate set 
of diffusion-dissemination activittes (listed earlier) and has managed 
to gain the strongest school board backing. This director had the most 
articulated strategy of change of any of vhs year’ s Title IV- Cc ) 
SERRCTINS , ; 
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CENTRALIZED: SERVICES AND TEACHER INSERVICE. . 0 ie \ 


The five projects which remain to be discussed are similar in that they» 

attempt to improve school system functioning by changing and upgrading 

the configuration of resources. * For the most part, effects of these 

projects on students will be indirect. In three projects this is som 

sort. of inservice activity; in a fourth project a mechanism is prcnosea 

for eliminating the need for substitute teachers. A fifth project . & 
.' proposes the organization of a regional instructional valetinten a thority. ’ 
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A brief description of each of these projects follows: a ey 


1. Shelburne Falls: ‘Proposes a multipurpose cnmbity resource 
center to serve nine rural communities through ‘workshops, a profession 
library, an “idea center" and other resources and vate to "foste 
the development.of our teachers, schools, and community. " 

Budget -- $115,600. 


£3 Aethehorca The ‘Attleboto-Swansea Project tnterser’ Nake a: 
replicable model for local sharing of teacher skills... Initial develop-~ 
ment has been in Attleboro with, replication“in Swansea.. This rather 
-complex project includes elements such as a self-generated catalog ia 
inservice options, teacher self-diagnosis, workshop’, team teaching, 
middie ‘school development, and a media center. Budget -~ $20,000, of 
which $16,500 is for professional salaries, 


8< Fall.River: "Promoting Reading linprovement in ‘Middle ‘Educa- % 


tion" involves intensive staff development and the ‘fostering of a new 
" support role for classroom experimentation with curriculum and staffin 
Budget m- $70,000, with $42,200 assigned to professional: salaries. - 


4, Worcester: "Project Sub" provides a mechanism ‘for inserting 
a variety of alternative programs and presentations in the classroom . 
‘in place of substitute teachers in two public and one private ‘high 
school. Budget -- S17 53885 of which ene is for professional™s.. 
salaries. ' 


5. Pittsfield:. Berkshire Regional Instructional Television 
Authority, funded through the Pittsfield Schools, plans to promote the 
use of instructional television by’ various theans including extensive 
inservice, assessment of hardware and software needs and resources, 
and group purchase. Budget -- $40,940, of. whtch $18,658 is for pro- 
fessional salaries. 


DESCRIPTIVE DATA 


Returning once more to Table 1, we note that this set of projects 
consumes: the siilallest share of the Title IV-C budget and that in spite 
of their rather grand aims and expectations, the average project re- 
ceives only. $32,000. More than other types, these projects are eelye 
to cover a wide range of grade ‘levels and they have the potential of . 
benefiting a‘very large number of students, sTbeyt indirectly. 
Finally, it should be noted that these projects re rather Tabor- 
intensive as far ‘ professional staff.goes,. the ees wl erogeer v 
suming: 70% of its budget in this way. 
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-Worcester's- Project SUB director: Ndon! t waa work with ne ‘adminis- 
tration, establish relationships, provide ‘resources, and evaluate what 
, ‘you are doing and how well it Seems to work ee you go along." 


on specific elements of problem-solving, we found the following: 


ar Diagnosis — nosis nf \ S, Nv : i ae i é = 
‘Nothing formal and nothing ongoing. Effort in this area was | ) >" 


minimal for most of these projects. ao” . ws | * 


* , \ id 
os inh 2. Financial Resources ae . ; ‘ 


Shelburne Falls expressed the need for much more. support: An 
uriderstandable, given the project's broad aims and modest budget. Other © 


projects indicated that funds. were adequate although there has not been vo 
much done to find additigpal funds. , Project »"SUB" depends heavily on * | 
, «voluntary effort which may be difficult to sustain over a ali a“ — 
years, a, : pov Te Pet 
te . ie a 


A é Apforinational “Resources 


: Although many of these projects are resource Vining by. their .' 
_ very nature, they had little to°say in interviews either ‘about the kinds 
and amounts of informational resourcés they had retrieved and made 
available, or about their strategies for searching and acquiring them. 
Two--projects depended on university connections, practicums, and credit- 
programs. One, Shelburne Falls, was severely limited by: budget. The 
Berkshire TV project indicated some frustration over the arrangement ° 
dictated by the state that. they spread equipment purchasing over the 
life of the project when effective operation of the rest of their pro- 
gram depends.upon having all the equipment in hand, - However, ‘this is 
more a matter of logistics than of information resources. 


oe 3. Generating Alternatives: for-Action ; am 
© &@ 
Only minor adjustments in operations were undeneaken by most 
of these projects. ‘Shelburne Falls has’ found that they needed to put 
far more stress on inservice activities and provision of services 
since the mere existence of a resource center.is not adequate to 
insuré its utilization. BerkshWfre TV ‘found that they had to put more | 
phasis on equipment maintenance and repair than they had initially 
fe anned for. Project "SUB" hag moved toward "packaging" series of five 
s tQ give more coherence, substance, and efficiency to their) teacher 
bets tion resource file. Such changes represent a normal response “~ |’ 
. ; to the needs of 4a situation and jreflect reasonable flexibility on the Ba 


and: deliberate effort to search, for’or test out alternative means to 
achiéve objectives or a move toward alternative OR ARENA eRe » — \ duis 


\ part/of these projects, “On the ey hand, none represents a planned ' ek 


DISSEMINATION AND OIFFUSION = 
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There has been either minimal or no ‘planning so far for’ y of these 


projects to extend beyond the originally designated ont ’ Project 
:"SUB" hopés”to get Massachusetts validation eventually;.fall River has - ey 
plans for’ reereenees at: “regional conferences and. systemwide release Ways. ne 
s 4 Mt 
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- CONTINUANCE EFFORTS =| ig? : Tr 2 oy" 


None of these sro iects had much ae say abete sont thuatth or support of . 
their projects on ather, than Federal funds. In Worcester, there has 

been some discussion between the superintendent and’ e ‘school board by 
about the possibility of picking up the project and project director 
has hopes in this direction, Otherwise,. the intervielimrevealed very... 
little information in'this area, suggesting that. the t 
high concern at. the early stages of a service-type activity 


EVALUATION ¥ it 


‘ - 1. The Procedure 


Three of these rorente plan to use logs.as a means of 
monitoring and. evaluating their activitiess Otherwise,’eValuation * 
efforts are minimal and perhaps not seen as a very high priority. 
Evaluation of general resource centers is admittedly difficult and the 


’ indirect nature of services to sttdents. makes outcome evaluation in- 


any ultimate sense almost imposs. ‘ble. “However, there appear to be, ; 
enough similarities among service projects to organize discussions on 


_ such topics as log-kéeping methodology and criteria for ShEv VCRs Oe 


evaluation. 


. besuits to Date Self-Reported by Directors 


Speed:. These projects tended to have some difficulty 
getting ne » partly because of the difficulty of establishing 
intersystem relationships, partly because of the neophyte status of 
some of the directors. Project "SUB" had a false start and changed 


both project director and involved schools. They now report good progress, 
_ however, with the exception of parochial §chool participation, Attleboro, 
as previously mentioned, has had scheduling difficulties. Shelburne =~ 


Falls reports overload. Berkshire TV/is not ‘certain, as of the time of 
the interview? how’ the project is doing and doubts the extent to’ which 
ITV is.seen asa regional priority he is "powers .that be." 

b. Numbers Affected: Although some projectsimight be more 
effective with a more circumscribed client BARTS EIAs there have bias 
no moves in this direction. 
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3 ‘ halite’ of Impact: Very little reported and Surely: too \ : 
) early to tell, n any case, HONS 


‘ 


Reflect s of Interviews and External ‘Evaluators . ath : 
‘ Regional ‘cooperatio on services, which is what most of these. 
projects are attempting, may be exceptionally difficult’in Massachusetts, ..- , 


especially without any state mandates or large financial resources to \ 
support it. In at. least two cases, the set, of communities being linked ° 


‘are more-or-less rural or small town and conservative politically and 


ducationally. Thus, any progress toward regional services is to. be ' 
applauded. It seems especially SAP EUnaSS that some of these service 
efforts are so enrny funded, . a 
CONCLUSIONS REGARDING: INNOVATION PROCESS IN ALL PROJECTS 
Most of what, can be said about process issues relating to this year's ; 


crop of Title IV-C projects has already been ae within each’ cluster 
analysis. However, ‘there are some issues which learly cut across “4 


projects and others which are elucidated by comparisons across clusters.” 


This section is devoted to discussing Sie: SGSE prominent of these. . : a 
TITLE IV-C PROJECTS AS " INNOVATIONS" ~~ - As : 


The underlying premise of the type of evaluation which has been conducted 

these last two years is that Title IV-C projects represent an unusual 

and perhaps upsetting break from the traditions and routines of every- 

day schooling. .They are new forms grafted onto old social organisms with \ 

the hope or expectation that the old will be somehow improved thereby. ys eM 

One question which ought to be asked, therefore, is whether or not these ° 
jects are indeed innovations rather than mere subsidies er increments 

inhexisting and essentially traditional school activities. « For an 

answer, we asked the directors to rate their projects along the dimen- 

sions of "innovativeness" indicated in Table 2. 


Most directors saw their projects as quite innovative in a. number of 
respects, Indeed, there is no particular pattern of differences among 


- types of projects in this respect. It should/be noted that more than 
one response was allowable fo this question én some directors checked 
as many as four categories while,others chosé only one. There was no 
respaliye from three prdject directors to this question. : , 
. A , 
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yA 
Projects 


| |. Centralized 


Curri¢ulum 
Services * 


Infusion . 


. Experience 
Fusion 


Pipes TN 


"tt is a very new and unique 
concept as far as I know" 


mnt is new at least for lassachisetts" 


"It is new at least for this schoo] 
district” 


t is new at least for the client . 
oup I am workjng with" ‘ 


"This project is.not really an innovation 


"in the usudl sense although there are « : te 
innovative aspects" - f NG 
i“ ‘ 
"thigyprogect is not intended to tee an) 
Ml Jn any real ‘sense" Pa 7 
t 4 
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Although there are exceptions to every rule, we found as in 1976 that 
there were a number of ways in which the process of project evolution | 


and management was similar, regardless of the type of innovation. . £ 
a % : = ; . ; 4 4 ; “= 
DIRECTOR'S ROLE a ' ; 7 ; 


Nearly all directors saw their activities fitting relatively within the 
four role mix suggested in last year's evaluation; namely: —" 


f 4 *, 
a Manager: (incl he planning budgets, administration, | 
‘Staff, recruitment, supervision, making key decisions) jou * 
* 7 + 


b. Facilitator: ‘(including coordinating, scheduling, con- “4 
sultant : project staff, problem-solving, team-building, linking to 
resources) , . ‘ ; 


7 ( 
4 4 i r ’ ‘ 
c. Communicator: (including dissemination, public relations, ant 
liatson, relating to ee media, speaking to otftsiders about the. .: i. 
project) ; fou, * 


d. Guiding Tight: (including writing, providing, ideas; 
content and procedures, instigators, trainer, inventing, ss sala 
designing, creating) ee “ 


Of the four, "manager" was seen as consuming the most time Rin al 
(30%), followed by "facilitator" (26%), "guiding light" (24%), and ‘ ‘. 
"communicator" saeee fy These differences hold roughly across all ' 
types of projects, although the “guiding light" role is dominant~fer 


-. curricular infusion projects ane sere lowgfor special needs projectg. 


_ RELATIONSHIP. BUILDING 


» 


As many others before us, we continue to note the basic stability and 
similarity among school district organizational configurations. 

Nearly all projects need some sort of support or approval from the 
district superintendent; the smaller the district and the larger and 
more ambitious the project, the more important this support becomes. _ 
It is especially:crucial for longer term survival and integration. \ \ 
After the superintendent comes the school board, and essentially the ‘. a 
same rules of magnitude apply. Advisory committees are usually likely ~ : 

to be another important element of the relationship network which must ee 


- be built and utilized, especially if there is planned to be parent and 


community impact and involvement. Ninéteen of the twenty-four project 
directors make specific mention of such involvement and for many it is * 
a core element. A fourth member of the relationship constellation is 


what cculd be called. the "administrative infrastructure." This-element s 
"ha ‘ , 
Paes 4 : 
eS oy 


S i . ‘ P ‘ . ¢ : ‘ 
» ds the most likely to vary from project to project; for some it will » 
. be the ieee for others, cantral office staff, such asthe director 


of pupil personnel services or the director of Federal. projects, or the 
business ager (a handful of projects almost strangled “in .red ‘tape 
this year);\for some it will be department heads (very important in most 
high school iconfigurations) The fifth vital element fin the raga da 
tion isthe\teachers, especially important at the implementation stage; 
some of this\year's projects succeeded quite well in gaining acceptance 
at higher levels only to run afoul of teacher schedules (which can some- 
times be a screen for*other types of resistance), teachers’ lack of 

* interest or ‘motivation, teacher conformance to pressure fyom other |. 

‘teachers,-and teacher skepticism about the value of the project or 


_ elements: of it. . at 


hor a few projects, notably in. the experience fusion and special needs %, 
categories, students also enter ito the constellation of key actors 

with whom.a satisfactory relationship must’be built. In some cases, 

good relations with students who have been perceived as “problems” 
enhanced a project's credibility with other key figures. In’other cases, 
an assumption of student!.advocacy may be. perceived as an act of Miena- 
tion from-the other more traditionaily powerful actors in the system, *. 


' Most project directors agree loosely onthe strategy of relating to 
Fa power figures, stressing frequent contacts, informality, and face-to- 
face discussions. On the other hand, very few are able to articulate and 
execute a comprehensive strategy of phased relationship et Ae 
cluding the use of many forums and media to reach and persuade both 
' formal and informal leaders of opinion.and gatekeepers. A 
: 4. . , Ww { : 
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* 2 PROBLEM-SOLVING, STRATEGIES © sg Mae: 7 


Projects were almost uniformly weak’in articulating coherent problem- 
solving.strategies. In many cases it appears that they are too 
immersed in what they, are trying to do that they pay little heed to 
‘how they are doing it and how. they might do it better. Possible 
exceptions to this rule aye Worcester's*Project SUB, Turner Fall's. 
Project*0.P.U.S., and Fitchburg's Proje€ét Competency. 
. . / hes o j 


For most projects, ongoing diagnosis and.rediagnosis of needs is either 
nonexistent or minimal, relying heavily on "informal" means. Resource 
ysearch was impressively comprehensive in one or two instances but was. 
‘generally nonsystematic and slanted toward predetermined solutions.’ 
The massive and‘readily acces sip he ate of the ERIC system are 
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BY On Specific fe he problem solving, we also find’ general wedknesses. 
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year's. or last year's projects, paenied Wenause this phase ‘of problems 


! . solving is thought to be Ture ee in the propesat development process ‘ iad 


‘preceding: grant pee Po 


aKt on f ov { 


DISSEMINATION AND DIFFUSION / } 


Thi “a in most cases local unless the logic of the project. requires a © o 
broader audience (e.g., MIT's WITS). With the exception of Somerville's 
"Outside-Ifside" and Fitchburg's "Competency" dissemination strategies a 
tend to rely on one or two types of media only and to reflect no overall 
strategy of orchestrating media, planning phases of awareness: and. 
interest building, or targeting key persons or groups. . Many“directors ~ 
‘admit that they have given little consideration to thfs area as yet, ’ 
‘assuming that within. the three-year cycle there ‘will pene be plenty 
of time for such matters. __, y; 


CONTINUANCE =|. es ee 


Similar comments could be made concerning dontgniance and apply regard~ 

less of the type of project, However, a strong argument’ can be made ° ae a 
for not enmeshing projects in continuation issues if they stilt have a 2 Te 
two and one-half yearg$ to run. ‘The reasonable attitude of most school» ‘ 

boards and superintendents at this point-would have. to be "let's see 

what. happens first--then come to us for more money and sanction." 

Nevértheless, at least one: project (Fitchburg): i's pee the way to 4 

full acceptance and Tibecus lon nte the schoo] district's budget and — 1. 
operating routine. ty . 


4! “ . ’ 


EVALUATION: ie | . 


A\thdugh tris area ad not phoged in depth in our interviews, it anpedrs 
that many projects are operating on very loose rutes regarding evalua- 
tion;. the picture is confusing with little common understanding: of 
methods , criteria, objectives, or uses of eatunt ve tformat i Few 
projects’ are self-consciously attempting to use in-process evaluation 
as part of a cybernetic system to provide openers pcre improve . 

- project ci Ue Furthermore, there may be litthe appreciation. of at Fw 
the potential public relations value of good avalaation 2 Poabang toward . et 
A project continuance ayy . 


OveryallsJudgment (Evaluation) ys - 


t is too early, to render any judgment on this year's Title IV-C 
program. Many projects seem promising and have project directors who ; dsp 
arg both talented and.dedicated and are giving the [State its money's’, ,»' * 
wowth.e, Nevertheless, there ig a danger that much of “this effort will © 4 

*he wasted ecause of lack of attention to the pitocess issues raised. heres: 
We recayn ize ‘that a wood dedt.af "protes’s" is just common sense and, e 
wi be done by project, directors, without articulation, without specific 
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Galitns, ‘and PUR ne: much. ‘awareness ; that they’ are sia Wotngh * e 

i, ‘Gt. . This is true for aspects of relationship building, for example; ena, a 

most pacvie who have had years of professional experience working in a 4 \ 

* school system know who the {mportant people are and try to get them of To 

their side. .The fact that. this is not undertaken as a conscious strategy “ 

«may not be too important. However, it Seems obvious that. process: issues 

get slighted if they are not planned for in a more-or-less systematic 

way. For example, severa4 projects would benefit from a broader search 

for informational resources; most people tend to reach only for what is ; 

either familiar or immediately avatiable either in their heads. « 0 = 

("experience") or close at and. “Similarly, there is no natural ‘tendency + 

to seek out and compare ialternative solutiens to a given problem or . » 4 

need; oncé the course is\ set, there is ee reluctance to change « ¢ # 

SRENDE possibly in minor \ways. i 


| 
THe 1976-77 MEC WORKSHOPS, FOR ITLE IV-C DIRECTO 


Nearly everything whieh has bean’ said in this year! s report reflects Ws 
what we found last year. In fact, this year's seminafs were guided in: 
large part by the assumption that inputs on various. aspects of, innova- 
tion management would be of great value in project improvement. It is, 
therefore, somewhat, frustrating to find such a mixed response to this 
year's early sessions. Sessions which dealt with the process o 
innovation in the abstract were particularly poorly received and’ were not 
perceived as being relevant by many of those attending. The report on 
Jast year's projects which was distributed to, all with the indication 
that it was going to be a guide tovthis year's training was uniformly. 
ignored by project directors. Those who read it could not recall any 

of its contents and it was clear that it had had no impact. On the ay 
other hand, some sessions were well received, melon a simulation 
exercise and a session on evaluation, 


A 


In regard to the general ea toward the problém-solving process 
which was taken, some felt it was too general while others felt that 
they had no time to pursue it with all their other duties; i.e., rather 
than being seen as helpful, it was seen as an added burden. A few 

_ participants also objected to what they perceived as ee pia i : 
nature of the sessions. ; 


In execution some-felt that the sessions lacKed dntdatity and that 
inputs from diverse presenters were hard to integrate. Above all, y 8 
‘directors expressed thevneed for more feared and many suggested 

that subgroups of directors could be formed around different problem - 
areas of greater relevance. Mt eee 
Reinforcing this point about relevance was the fact that most: directors \ ar. 
expressed appreciation for the field interviews upon which this report iy ‘ 
was ‘based; they liked to have the opportunity to review the process of © : 
their: projects in’ depth with a’concerned outsider. This also suggests 
that the "process" issues~and questions are.seen as highly relevant — 
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~ when they can be placed in a speci aha self-relevant contexe, 


, Looking toward the next year. one director suggested that Feknevtens and ney 
‘ field Visits might precede sessions and be used as.a basis of clustering. 
“ene that this year's mode of “presentation according to project types 
l.assist in that. direction. One. director also observed that it would , 
‘ make more. sense to provide much of this process input and en: for 
prospective directors Renee to, proposal Senn 
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THE, TECHNICAL ASSISTANCE ROLE OF ERC. 


N 4. 
THE GENERAL VIEW > 6st a 


“Within the general rubric of the donrtede’ Education Centertanavinene’, ae 
of Education approach, Educational Research Corporation (ERC) was 
“asked by MEC to participate in. the project and to lead key work «| 
sessions. Havelock's ‘report, Chapter’ One in this. document,‘ is general = 
while arranged to highlight programs and issues. ERC's report s : 
a task orientation as befits the charge to them. Based upon the needs - F 
of.the project directors, ERC led group work sessions on evaluation pr 
techniques and on information exchange processes. A ee director 
+ Ge survey was also conducted. ; ‘ 


The purpose of surveying project directors was to get a preliminary 
- -dndicatioff of problem areas Title IV project directors have ‘dealt with, 
_ were dealing with, and expected od ll to deal with as their projects 
progressed. The ques tionnaire (see ‘Appendix B) therefore asked 
directors questions relating to Communications, project objectives, \ 
project: operation, evaluation, and training. | 


ERC provided two forms of feedback on the data contained in the survey. ‘ 
The first, summary outlines the reported needs for future training of © 
project directors and the second discusses results on the remaining 

topics covered in the questionnaire. In reporting the latter results, 


i Y “the report primarily focuses on the responses for Title IV-C project 
| directors as opposed to Chapter 622 since evaluation of Title IV-C . 4 
j projects was ERC's primary reponsibility. . However 3, in the few in- , t 
/, stances when differences between Title IV-C and Chapter 622 prajects 
sige were substantial, notations are made in the summary. \ 


TRAINING SESSIONS \ 


P 


ERC had several levels of involvement in the training of Title IV-C\y 

and Chapter 6a project directors. For some training sessions, ERC \ 

staff conducted their own seSsions. In other sessions, ERC staff 

participated jin training with MEC or State Department of Education : 
staff. Finally, at a few sessions, an ERC staff member merely ob- ; a 
served sections of AL een hale to obtain background information 

relevant to other aspects evaluation responsibilities such as the 

booklet preparation. Table I summarizes ERC*s involvement in the 

bigs, of ‘project directors. 


The primary effort of ERC's involvement in training sessions’ for pro- ‘w 
ject directors went into the preparation and execution of Sessions 
which ERC conducted,specifically, the January 20th and May. 12th 
sessions. Involvement in the remaining five sessions is either 
outlined in Table. I or covered under the topics which follow. 
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Mice: : . = . . “i ; ; ; ki ei ; 
<The January 20th Workshop , i ; ; aes ; 


1s f 2 a \r ‘ Bs - i 
ERC's conducting of the January 20th afternoon workshop was in re- | ‘ 
sponse to the.need expressed by many project directors in the fall sf 
questionnaire survey for assistance in evaluation. The afternoon - 
' _ meeting opened with a géneral session on evaluation to discuss the 


o.. basics “of. evaluation, design and to provide some fundamental. information 
Pal on devéloping: questionnaire and interview instruments. (See Appendix 
{ B for the Agenda.) “After the general. session, participants chose one 


of three interest sessions to'attend. The interest sessions covered 

-- (1) item writing for instruments. (2) use of evaluation results, and 

(3) practice in interviewing. Three smaller sessions went into more ~ 

-. _., ‘detail on respective topics. Finally, the entire, group reconvened , 
4 . * © fora wrap-up session where participants were requested to complete 
ar - evaluation forms on the entire “day's gs MA The,evaluatior forms 

6 demon rate 6 
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- were then us@@ as training devices t good and ‘p r in-. 
'strumentation practices. . 4 y ; 


_ Throughout the afternoon, the varios trainingxtopics presented in the 
e discussion were supplemented with hand-outs for participants’ to keep 
and use as they proceed with their projects. The training forms. 
included forms for Sources of Informatton/Data Matrix, Information/ 
Instrument/Item Matrix, Some Techniques for Interviewing (a do's and 
Ydon't's list), Objectives/Activities/Evaluation Organizer Form, a 
Selected Bibliography on EvaluationsRetated Topics, and others. 


As, a follow-up to this training session, ERC compiled the results of 
the evaluation form to ascertain participants' perspectives on the © 
relevance and usefulness of the sessions. A sucmnery of the data 

‘ appears in Appendix C. fle 


The May 12th Workshop : Se Ei ’ %, 


The second training session which ERC conducted was directed solely to 
Chapter 622 project directors. The purpose of the session was two- f 
fold: (1). to have directors exchange information with each other about 
their projects, and (2) to assi§t them in developing a summary report 
which the State Department of Education Title Iy-C staff would: use to 
compile a directory of all 1976-77 Chapter 622*projects. 
p To achieve the first purpose, each director at the session gave a 

summary of his or her project, emphasizing resources--materials’ and 
people--that other directors might find useful.in future work. © Much . 
exchanging of information and resources occurred at this point in the 
training. To achieve the second objective, the ERC group leader pro- 
vided. directors with a suggested outline fo™ writing a brief summary 

of individual projects with emphasis on providing recommendations 

aimed at various levels of potential readers--future Chapter 622 pro- 
ject directors, Department of Education innovative project supervisory 
staff, and individuals wishing to replicate existing projects. 
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INDIVIDUAL CONSULTATION WORK, 8 | ae ne 


At the Novemberé 17 th. meeting and throughout the remainder of the. eee, 
“ERC encouraged project directors to use the staff expertise at ERC ; 
to assist them in evaluation, -or for “that matter, .a vise problems - ( 

A 


they met as they proceedéd with their projects, hough ERC did not *, 
have the means to visit individually the various projects, members ° 


“ were available at training sessions and could be reached by we 


¢ 
In addition to formal consultation work on these four projects, there 


mail to discuss individual: “project activ ties. : 

Four proj directors took full. sdvantaue: of fhexa services ie: did 

send materials to ERC for critiquing. Usually the matertals were 

evaluation designs for which project directors wanted input as to their» 

viability and rigor. An ERC member responded in writing to each of ' 

these directors With suggestions relevant tos the specific material: \ . 


given to ERC. In oné case,there was-a “phone: Followup. toa the written 


communications. ay MS = ee Ue 


1 
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were numerous instances where an ERC staff member discussed individual. 2 “4 


project concerns when project directors met. at training Sessions. , These 
information discussions became much more frequent after the January 20th 


.traihing session when-project directors had a better a of 
_ the services available at ERC. 


Consumer of evaluation. ‘That is, project dinéctors shir ota 


Finally there’ were, of course, individual project- -related iesktans 
answered. during the course: of small group training on January 20th and _ 
February \Sth. - . , ; 


# f 
c 


EVALUATION BOOKLET | 3 i's 
b % , . 

The final major service grovided by ERC in the evaluation work of 

Title IV-C projects was We preparation of a program evaluation guide 

to be’ used by future Titlé IV project directors. Mostof the people 

directing the newly-funded innovative projects have little experience ia? 

with formal evaluation of programs and need guidance, not particularly. tA. 

on how to evaluate thoroughly a project,but rather on how to be a 3 # 


O become experts on evaluation, but they must be familiar enough with PS 
evaluation and with designing a rigorous program so that it jJcan be 
properly evaludted by specialists. 

The booklet, which is the next chapter in this report, covers two basic | 
topics--the development of operational program objectives and the use \! 
of an appropriate experimental design for a project. The objective 

of the booklet is to have project directors initiate a a ta 
will be both internally and externally valid. The need ‘for n- " 

ternally valid project ‘obvious, but the concern for external vali- 

dity has special meaning for projects funded by Title IV- -C, ESEA. , be 
These are all innovative projects which should be atgempting to solve : 
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~, “programs established at their local level, their goal should be to P 
x» have other educational units_adopt or adapt similar projects. ‘It is .e NY 
~ ‘therefore vital that new project directars consider the external | 14 
. validity of their projects from the beginning. The booklet therefore « 
proposes to assjist the new project director at the beginning of his . a 
or her project to establish a program which will meét.. long-range ob- 
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general seach Rete Once project directors have successful, - 


—* , jectives of Title IV funding. ae 3 ste 
; ” a \ fF F 2 ; 4 ‘ 4 . f 3 } ‘ 
'. » SUMMARY OF QUESTIONNAIRE RESPONSES | ~ ia lg s eee s 
. During the ldte fall, 1976, ERC sent. to ‘project directors of all . - ‘ 
Title IV and 622 projects in Nassachusetts a questionnaire. [ts pur’ a ae 
pose was to uncover problems the directors of these projects were 
having insorder that the directors might receive specific training 


and “technical services. The potential problem areas which pri- |. ey ale OF 
marily concerned ERC aaa objectives, project \ ee 
« a w 


operation, and project evaluations . 


& a ° i 4 ‘ 
Most of the Title IV and 622. projects were in their first year of -\ r 
Operation; only. four of. the 38 had been operating for over a year. 
The current project director had been the first to-suggest the idea . 
for the project.in 26 of those. cases; seven had become involved first * 
aft the pre-proposal stage.. From these figures and from the nature of .- 
the project directors' comments,.we can safely say that at,the time 
they responded to ERC's questionnaire, they were committed to their 
‘projects' success. In addition, it is clear that they were $ttuggling 
with the:need to establish effective communication linkages within | 
their school systems and communities, to clarify their project's ob- & 
» jectives for themselves and others, to translate proposal goals into : 8 
the implemented realities of project operation, and to provide for °,. : 
some objective evaluation of the project's progress and success. * 


m 


The following focuses on findings regarding the evaluations ‘of the 


\  Fitle IV projects; 622 project findings are included ohly when they \ 
Significantly differ from Title. IV findings. of 
: . : _ & 
; ‘ #, 
COMMUNI CATIONS oe eae: 


Most prejects involve communications with or dissemination of.infor- , \ 

mation to several audiences both within and outside the schoo’ system. .* « 
* As the following chart shows, the majority of project directors are 

"satisfied" with their level of personaT:contact with al] but the 

parent categories. ‘ “ } 
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a Ag PERSONAL CONTACT OF get 
Re . | TITLE IV PROJECT . on Satisfaction ° 
DIRECTORS WITH: \ oe - $i with Contact 
. . Satis- WantMore 
\ fied Contact 
Superintendent | 
Other central office _ ‘" J . <a 
eK administrators 3 ; 
: School board members : 
* | School principals 
School spe@lalists 
Non-project tea@hers 
Project teachers - 
Participajing students 
| Parents of partici-, : 
pating students 
| Other parents P 
Town officials 6. 
Public media people ~ . 7 fo 
Other (specify) - . , 
university people oe 
‘ community agencies 
‘ \ : 3 
Fairly large percentages of project directors also expressed a desire 
for more personal contact with school board members, non-projas 
e teachers, town officials, and public media’ people. This reflects their * 
wish to broaden the audience affected by an informed about their* 
projects' activities. It also indicates a high level of “awareness among . 

; project dire@tors of the importance of personal contact with people : . 
other than those directly involved with the project, especially people . 
who might influence their efforts to obtain funding and support for eee 
the continuation of the. project: beyond its initial funding period. 

ff 
- « However, a number of project directors said they had too little time 
to make all the personal-contacts they would like. Sometimes ag 
was because they were Jocated in different buildings from their poten- : ; 
tial contacts. _ Sometimes the project director found his/her regular 


45s of 


- ' ‘ 
‘vy ; ot ade wt y eo 


{ i ie 


‘| iteaching duties did not allow sgh ine fr the project. Besides 
Wr - time, the other major obstacle to establishing more personal contact 
with the above audiences cited by several project directors was lack - | 
Tee F¥est among, those being contacted, who evidently felt that Title 
IV rafect's goals were “nat a priority" or. "extra-curricular." | - ‘ 


AY) reporting project directors use other means besides personal con- 

tact, however, to inform these and other audiences. About half have 

used newspaper-articles; all have used memos or newsletters or reports, 

Also cited were such: diverse means as cable TV broadcasts, slide 

presentations, inservice programs, and speeches to alumni or Phi Delta 

‘ Kappa. Ten project directors expressed satisfaction with the effective- 
: ness of these communications methods (other than personal contact); six 


Pa 


\were dissatisfied; and seven felt they were "good" or "adequate," Pe 
In summary, then, although somewhat hampered by lack of t}me, the \ - wan 
majority of project directors are satisfied with both the’ frequency tog 
of their personal contacts with most audiences and the effectiveness 
of their other means of informing others about their projects. In , 4 3 


general-the audiences with which they’want ‘more personal contact are 
parénts, public media peophe, town officials, schoo] board members, 
and non-project teachers. - ‘ ‘4 , 


‘ PROJECT OBJECTIVES 


¥ 
“ag 


Only eight of the projects (fewer than a third) reported making modi- * 
fications in their projects!-dbjectives since last summer. All‘de- 
cisions to modify the projects’ objectives were made by the projects' 
staff, directors or arficipants . Only two said*'the school. adminis tra- 
tion helped in the-modification process. Although the changes were 
varied, ranging from adjusting expected outcomes for students to 
budget changes, in general they reflected -a,delay in the start of the 
project and/or the scaling down of objectives to more reachable di- ° 
mensions. Seventeen project directors reported no change in their 
“ objectives. About two-thirds of the project directors felt their - ; 
- - Objectives were "very relevant"; one-third said they were only "relevant." 


When asked whether there were groups.with whom they deal who seem not . 
to fully understand the objectives of the project, slightly over half 

of the project directors said "yes." Most of the groups cited 

(teachers in other departments, central administration, parents, com- 
munity leaders) were again those without direct vol eden in the 
project. The reasons for their misunderstanding were given as poor 
communication, resistance to change in general, or specific-territorial,, 
worries causing groups (such as principals, administrators or non- 
project teachers "8 feel threatened. Some of the lack of communication | 
was intentional on the part of: the project director--a wish to dissemi- 
nate informatign or inform others of procedures only after the con- 
ceptualizdtion of the project was more solid. This, of course, reflects 


» the fact “that most projects have been prerating far less than a year. — ‘ 
# a res \ Fr . . 
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"Everything is just taking longer.. than I paOUghs” 


_ with some aspect of the evaludtion. Other. answers were diverse, 


PROJECT OPERATIONS PS | 


a + ee 
# il , 
i i a | uy Wy ' ‘ 
aancal ‘ : ' 
¥ ' ; 
‘ ’ + : J ‘ % 
; / * i ‘ ' ° . SN 
4 d ; , Ps ; ae, 4 iz 
4 ’ My ; 
% ~~ 3 4, : . 
| Lg eR my BMS : fo ne “ : " re 
«a bh * 4, ; E : ; 4 4 . ce Pe 
vs Pond is : ek 
‘ 


iy A a . $ 
weld eg SS =" « 4a.” 


Seven project directors’ ‘Said there were not project objectives which’ aay 


_ they were having ‘difficulty achieving, : The. others had found difficul- -. 


ties baséd on various groups’ resistance to change, the difficulty in « 
getting people to follgw through from-the planning’ to the action stage, . 
a need for a,change in ithe approach, to: or conceptualization of an. A 
objective, or the simple: fact that, as one project: director Sets: a8 Pen Mines 


' 
kisser 


When asked what project objectives they aitteipated having “steely ’ C 


* achieving in later stages of the project, ten of the Title IV project _ 


directors replied, "None." Three more indicated they foresaw ditficulty. 


ranging from resolving legal and ethical issues to. Ey, for , . ry 
institutionalizing the activity. G 


rd 


w 


In general, most seaifuas listed obtaining project approval obtaining 7 a 
appropriate facilities*, obtaining relevant materials, and*obtaining « 
organizational support as their easiest activites, Recruiting appror 4.) 
priate staff was listed by about equal. numbers-as creating no difficulty: 


_ and’ creating moderate-difficulty. , Obtaining funding--bagh initial and 


continuance--was also listed as a moderately to greatly difficult area 

by eight projects, The following chart shows the details of responses. = ape vy 

when project directors were asked to rate their ease in hand1 ing, da o ht 

areas of -project operation. sare 
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*This was not among the easiest for 622 projects. et He “\ 
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i fee SE eS eae, ES ee Great, or very 2% 
aa “No-or little Moderate: great: +e a 4 
“a4 difficulty difficulty: .diffitulty ©, Not ; | 
Ain handling = in handling, “in font éreleyant a 
Mae ‘doednind! inftial funding “+ 12 a , 7? 
e taining ‘project approval * i ara ie: Sy 2 \a ‘. ot 
iN iting appropriate: staff. wf 9. : no’ pe A @ *f te ‘ 
; eduling project staff 7 ey f 2 <> + ae, a 4. hy 
‘ige (| Scheduling studedts. 2 9° (WV, . ae ire: scares 
Recrulting participants, 3 he i a ee a ie i (ae ; 
Retaining participants a’ 15 2 ie ‘ i ye . . 
LObtaining arroprjate Pee Pa "gp eek SOR EY go ® 
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ob¥aining retvant m nét?Js. ch. oe ; at ie 2 a ae 
| scheduling’ traintng’wkshps. Je 13 ee eT Ps 
Schpduling patent meetings’ Bt gt tng Z': Jf + Js, 2 
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2 VR “About Kar? oF the pridact:; St alyctr sid their projects! “najor “Strength 
2 saan the quality of the staf particularly in.their ability to work 
EP a team, Over a fourth cited thé strong support of their communi ¥} a 
' ee Zchoo!. system as a major trength, everal. also mentioned godt ‘ ‘s 
ae . communication with teachers as a strength. Organization and planning f é 
Mites Sok ceili as pcre. by only two..proj cts. The various other me van i 
oo eo sponses réfle iy the’ a directors' belief that student interest 
- was high, . several nee ere being. met at. once or ‘the“changes were. \ . 
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? ; at re 
While all projects plannedato conduct’ an internal evaluation, many 


’ 
y . 


expressed uncertainty about the methods to be used. Eleven had begun | 


the evaluatién (mostly in the sense of havin Stven some King of 


pretest); fourteen had not. ; 


We B #89 asket’ project aivhator hehe buscild (in their own terms) ° 
felt their projects would-be. About half said "very successful, 


sh e ‘the remainder. cheeked Pame hat successful. ae 


Severdl salient points are to he gleaned from the results of the . 
_ questionnaire working with the validation of innovative projects. 


A 


* 


also suggest certain areas. ae aeaeeite and technical servicgs in the , 


* ee 


<6 ‘Project diréetors: ‘feel bodies with their level 
‘* + sof personal contact With all but parents. “Some 
; (project directors.also,want more persohal, contact 

= with school board members, non-project teachers , 

_ town officfals and public media people. The main 
reasons cited for limited personal contacts were 
lack of time and Tack, of interest/on the pars of 
others. , ’ 

‘@° -Fewer than stethind of the projects have nodi ied : 
their CEUECEINSR: i 


: @ Over half of the project “diraetony? ‘felt there were 
groups that did not ully understand their ob- 
- Jectives. ee? seh, Shar ae 


» @ Qbtaining fundihg seems to be aiarea- with which 
» } a number of projects have expertenced ali 
| , to great difficulty. ie 
° 


About half ‘of the project directors feel the 

salad of their, staff. and/or their ability to. 
work together ig: their project's major strength. 

Commun i ty oe was ang: as a major ay 


ue by/ 25%, ; 
; : yf » * ; ‘ aN : 
@ °@ Many. projects’ expressed uncertainty’ about the 
Sk fhethods: of internal evaluation ‘they mot use. 
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Ed@cational minin dot has seecenltshad under the direction 


of Merrimack Educati 
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, project divectows: training. directors, sk ovtdtng individual congulta- “ 
“tion, and producing an evaluation booklet for future dirgetors., These - " 
“ services were provided as part of the total‘effort of evaluatjng Title 

IV-C E.S.E.A. projects, which also included Chapter 622 projects. +The 

effort not only focused .on existing projects with their immediate needs 

.and. problems but also looked ahead to assist future. projects and 

, directors to meet better or to avoid some of the difficulties experienced.”,. i. 
* with. current programs, ERC therefore hopes that if has been af service oS 


to current programs and wil] also be of service to future, programs ey a 
through its efforts this past year, The hext chapter reproduces wer SS 
another product 4 ERC used during the training BeeIONE a Oh) saerg 
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; > © © GUIDELINES FOR DESIGNING AN EVALUATION PADCESS ie 
&. THE NEED. TO PREPARE FOR, TESTS OF VALIDITY  { 


oat 


. You have, just. thought of.a marvelous 


art appreciation to elementary stude You've talked 


h 
with a fewof your fellow teachers andegotten: them en- Mi sas #2. 


worthwhile that whgn you informed your principal -of the 
‘plan, the principal ‘suggested, that you.develop a proposal 
for funding tHe program through Title IV ESEA. Your oa 
proposal was accepted and here you" are with a dynamic 3g 
idea, enthusiastic co-workers,,and.even a source of : 


thusiastic about Nr approach, » Your jdea sounded sq”. 


funding. How do you turn that sensational idea of yours Ne ; 


into a viable program? How do you‘convert that enthu-- 
*siagm and.support into a solid, realistic aspect of the - 
, & ‘school curriculum? And suppose your program does become: .° | 
part of your, school's; curricula, how witl) you really 
know if the program does: in fact, help elementary stu- l es 
dents better appreciate art? And suppose you do deter- 
mine that students in your program develop: a better 
‘appreciation of art, will your program do the same for . 


students in other schools? 
s 


What you want to do is ‘turn’ ideas into workable programs and more, Once dp 


you have set up a program, you will want to be sure the program is 
actually doing the things gt is meant to do and you may even be in- 
terested in finding out if your program would work in other places... 
In determining if your program is doing what you want it to do, you. 


. are asking that your program have internal walidity; in determining 


the applicability of your program outside its present setting» you are 
$0 ie that your program’ have external validity. The first considera- 
i » that of internal validity,is important to determine because you 

wil] want to know -if your program has achieved the desired changes in 


the learners with whom you are working. The second consideration, — 
. that of external validity, addressesgmore general educational, con- 
“cegns. me it is helpful to establish programs which meet, your 


specific se ooMor district needs, it, is more desirable to'develop 


« imaginative solutions to educational problems common “to many com- 


munities, ratner than to fund projects having meaning to one community 
only. It is thus important for you to design a program which (1) you 
can, prove to be successful (has internal validity) and (2) can be 

used by others (has external validity). The information contained in 
this booklet is deSigned to assist you in gonverting your ideas ‘into 

a program with internal and external validity. To achieve this goal, 


. you will want to concentrate on: 


e@ developing appropriaté program objectives and, 
@ conducting your program within an experimental franiework. 
- 51 - 
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new way td teach. etre eno ae ae 
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_cdectives, and evaluating educa 
college libraries. Within this booklet, we provide@Some of. the better- 
_ known materials: usd by educators so that’ you may address your specific 


* your district is relatively large, you may have planning, res@grch. 
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Before elaborating | upon the dethils of these "bio areas, {t is, impo tant 
te“consider all.the resources whtth you can call upon for assistanCe 


- if developing your educational program.’ This bookle¢¥ is- only one 


resource you shou}d be te There is a-host of written material on 
developing educational programs, writing program and curriculum op- 
tianal programs availapde in local.or  . 


* 


problems. i 
Another area of, assistance is your Title IV ESEA regional specialist 
of the Massachusetts Departm@nt of Education. -’ The specialist ‘can hel 
you with.specific problems or can refer you to other helpers. a 


ie Bis ’ 
* Another area of assistance ifcludes professionals dealing with educa- 


tional research and evaluation. People with experience in these areas 
should be able to offer you assistance in helping you show that your 
program works, Such people may)be. in your jown school district. If 


and evaluation units With people who could/suggest approaches to take 


- in developing your program. Your district’may have indivjdudls with. 
* relevant graduate course. work or formal practice*in program design... 


If you do not know of people with appropriate skills, your principal 
may be more familiar with the-talents and experiences of people in the 
district. © ; Be pas ee ra 


, 


\\. There are two sources of outside assistance. .One is ‘the university ‘ 
Bis or ‘instructor; another is the private educational “educa 


o find appropriate assistance at the university,contact the educat 
or social science department of your local colJege or university, state 
would be interested in consulting for youd. Your local educatjonal , 4 
collaborative is also an excellent place to go to assist you'in finding 
an evaluator. The collaborative has contact both with local university 


staff and with reliable education companies which special4ze in the type 


of evaluation assistance you need. You may’ want tO contact your dis« 


*" trict person in charge of federal funding for lists of yi a ek 


evaluators. For example, Title I and special education of. ces both® 
have such lists. .If ‘you use outside professionals, it is likely that 
there will be some sort of fee involved for consultation work. 


The key to using any of the above resources in helping you design an 
effective program is to solicit help early, while your project is 
still flexible enough to incorporate those elements of planning and 
‘design necessary to ensure a well-developed program. It is too late, . 
for example, to learn in March that,'the students :participating®in your 
program since October should have been pretested for “baseline data 
before they participated in your progam. , 


cal 4 a 5 


onal 


‘your needs, and then inquire if, there are appropriate professionals¢who ., 


i) 


“ 


} 


‘ . 


te ye 


Regardless ' of your sources. of deetetange:: you will want to become as 
well versed as‘ possible..in the techniques necessary for: developing: + 
» +> and cofiducting your ‘new iy ean so that you can guarantee that your » , 


& ideas are being accurately translated into:a viable program. While 
‘you may rely on others to know the intricacies of why one particular 
_ °° test. is used over’ another. to Bae vent your learners or why, certain... 
"| ‘statistical procedures are applied to the data generated from your. 
project, you will want to be familiar with basic considerations: made 
about your"project ag it is translated into a sound experimental model. 
The next two sections are designed to give- “you pial ‘information neces- 
sary for, developing sone new program; — 


’ ° 
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_ PROGRAM chucet ives” ae es OP ee 


* you have already written ’a ‘series the pro- 


| 
‘| program objectives i 

* , + posal which-was funded. -Some tiie has elapse since thos epyec tives 
% ’ were written and now-as you start to implement, your’ pro ram, y 

should have a’ slightly potter understanding of What. it/is “that ON 
i want to do and what it-is that you ‘can do with your program, It is’’ 


now. time to review those program. objectives, with the help of your 
evaluator and any other asst$tance available to make sure that the. 
objectives state what it-is that you reallywant to accomplish with, 

« ‘your program.” To siiustreye by using the example of art appreciation 
for : elenfentary students, you would want.to be more concrete in specify- 
* ing. OnJEC elves than he a stating: ; 


.P OR: Elenentary students should learn to appreciate art, 


While -this may be a a of the program you would, have. to define ex: 
actly what you Wish fo accomplish with the program, What you wish to. 
hy ‘| accomplish would be stated.as your program objective. For example, 
fied ayou;migh® want students to recognize various art forms or you. might 
swant students to uge specific art processes and techniques to express 
: ‘a themselves. Your goncept of ghe program may not even directly relate 
i ' to student achiev lent. MO4 may be«more ‘interested in training y 
teachers to increase their comfitment to art instruction. If this is 
tHe case, an bjective might read as forTows : ; 
BETTER: ' Teachers will develop an art unit based on each of 
‘ : the ten forms of. art discussed in the seminar series, 


\Whatever the objectives end up. being, they should *resent a Clear focus 
ae of exactly what. it is that you want to accomplish with your project. 
our objectives must be a realistic statement of what you expect to 
happen with the program. To understand what is realistic, consider 
, " each of the following areas as it relates to your program. 


3 If we assume ‘that you are|'the mi te Title IV" projecf director; a 


‘ ea 


i ee ' / 
- .0n the one hand you will wapt to ees your idachen Gaatiemiey and the, 


*novative-so that its success would be~a valuable contribution to the ee 
ealieat process, but, on-the other hand, you do-not want to set - ey 


“The availability of participants for the program, ‘ * a i 
The length of time! and level of ' intensity for working « 


The.cost of the prosrap. 


La) 


* : a ” 


The needs and abilities of your Jearners and _ < oF ye 


participants. : ; 
! 
The staff you will have available to you in your «“ 
prograr . mig : ™ 
wietencstiniet and materials avaitabie or Vikely « 

to be available to,you.. - 


with your learners/participants. | ° 


' 


The, ability to determine if your objectives are fe ey 
being ScConpETSheds we 
an bi A i. 


* i oi objectives so high that it would be impossible to aeriaye thee 


er the conditions within which you will have to opera 
> ae 


in terms of each of the above seven categories.” Ask these questions 


j _ Therefore, as you write each of your program objectives , think of it 


of each objective. (The phrase "Do I have" can Frequently be , 
substituted with "Can .I get".,.). 4 
2 > 


: 


What evidence do I have that there are te Rae. need 
this training?’ Is the objective appropria 

level? interest level? knowledgeability of my partici- 
pants? ¢ ‘ : 


Do I have sufficient teachers/lecturers/trainers? ‘Are 
they qualified to work with this type of, participant? . 
Do they have sufficient interest or cor titmgnt to. TAs GE 
accomplish the objectives of the progra? Do'I need < 
ROMT SEED PVE a Can I get tiis Bee SP Heees 


Do I «have the physféal space for conducting these 
‘classes/holding the seminars/building this equipment 
etc.? Can the participants get.to the program site? ¢ r 
Are there teaching materials available? Do I or my 
staff have the capabilities for developing needed 
training materials? ‘ 


e for the we 


‘ 


‘ “Who are the people likely to be utente in the program? 
’ How will I inform them? ‘How many people wil] the program 


a B® as 


eg 60 


a4 »=onth? -one year? - 


' “attract? Is thts more appropriate for a volunteer or ‘a 
a captive group? Which type of audience will sal 
available for my program? 


5. Goh 1 long do I have to accomplish the objective? one - 
three-years? How frequently will: a" - 
-I be able to work wit the participants? Will.this 

bea reasonable time accomplish the sid sa ee 


o How ‘much wild ‘it cost for the appropriate facilities? ; y “s 


gee Ee staff? participants? time? 

7. How can I test this objective to see if it is accomp1 ished" 
and to what degree? Has the objective been abates in * 
such a way that I can measure. it? 


ion that you have answered each of these questions , for each of your 
objectives and have modified your objectives so that your responses 
are. realistic, think about your objectives and: -your answers ih terms 
‘\of other schools, or districts. Would these objectives be valuable 

and: desirable for other schools or districts or are they simply solving 


a unique pee oh for you? Obviously you would want most or all of yqur . 


« Objectives to be objectives other educators would find useful for 


solving: some of their problems. Look back over your answers to each of” 


the questions. Are there certain features-- availability of facilities, 
characteristics. of ‘participants--that would not likely be found in 
other -classes, schodls, or districts? If you answer is "yes" to this, 
then you would want ‘to consider modifying your program so that your 
program is no longer a unique answer to a unique problem but is a re-. 
producible process,addressing a general educational concern. 


One final caution in writing your objectives is that you /should have 

a reasonable number‘of objectives. If you end up with sixty objectives 
' for your program, you either have unbelievably extraérdinary resources 
available to you or you are writing your objectives .in too much detail. 
Your objectives are probably too specific. To use the art apprecia- 
tion project, an unsuitable program objective would be: 


"POOR: 80%, of participating third sibadavs shakl success ful ly 
RIE Michelangelo’ s "Moses" as a stulpture. 


| To correct. _the avoeten of overly detailed ‘objectives, you would . 


generalize ‘from Hs and similar objectives to one program objective 
such as: 


BETTER: Students will identify ganteis art forms appropriate 
to their grade level. - : 


_ goals so they must be specific- enough to be able to measure, .but on the 


‘The point to remember when writing seh diiahines is that -they. are. 


just that--program objectives. “On the one.hand they sare not program 


other hand, they are not curriculum objectives which would be very 
specific statements such as our "POOR" objective about’ Michelangelo's 


‘» "Moses." Program objectives fall somewhere between goals: and te ee 


' because of the great diversity of .Title IV programs, Each set of. | | 


' thé program. It is. sometimes hard to remember with programs of our own, 


, Before discussing these features, it is helpful to know just why the 


curriculum objectives with regard to specificity. They must be general 
enough to state concisely the-major expected outcomes of ‘the, program ' 
a also Specht) Cc enough 4 fo represent a measurablegoutcome, . 
i * - y 


PROGRAM DESIGN * 


Now that you have your objectives clearly and appropriately stated, 
youfare ready to design the Operation of your program. . You have two , 
primary a aa At: paren the considerations for. your a t% "% 
They are: ™> ts 


{ 


| eo. Designing the’ program to meet its objectives . : - ‘ 


@ Designing the. program to test _the,dearee to | «= 
which et . met its ectix e 


Designing your shenran é meet its object ves will not be covered here » 


within the design of the program are certain conditions and procedures’ ~, ! 
vital to the testing of the program to determine to what degree if\has 


program objectives would be very different. However, ante 
met the objectives. 


testing. of the program is as important as establishing and conducting 


ones we have developed and monitored through the ares nen 
and start-up phasgs,» that program objectives may not be ved or 
that they may be achievéd-dye to other factors, such as student. motiva- 
tion, rather than the program itself. We must not assume a program 
will be a success--those not ‘committed to the program certainly will 
not assume success. We must prove success. This need for proof is 
vitally important for a Title IV innovative -program and an absolute, 
necessity for a program which we want other: iat or systems to jd 


‘replicate. 


Proof of success is a difficult task in the educational field. There 


(are, however, certain elements you can incorporate into the program’ 5 ; « 


\des ign: “to provide evidence that it did produce the changes it was 


intended to produce. These elements are: 


— _ volved in the program. This 
ar line data which tells you’what your particip ; us Oe 


‘The reason for. complicating your phogran with ahinas like a comparison Ny 
group and randomizing learners/participants jnto comparison and 
‘treatment groups and so forth is to reduce the chance that something ° 


td anveffect explained by other factors. If a change is observed at 


We will first. discuss the factors which must be controlled’ so that your 


*See appendix for guidance in testing if you are going to use 


‘. ‘ i} : x” = 
,.@ Comparison group -- a,group which does not receive. by. 
ot treatment Chetroetion in your program as opposed PA 
Way _ ta, the experimental (or treatment) group which does 


f 

fh | participate in your program and with whom you are. —. 
et os “ trying to:@hange in some way. (Comparison groups te 
y ae 8 referred to as contro] groups, ) 


@ Randomization -- the assigning of learners *t9 the. & \ 4 
| comparison group or the experimental group with ‘equal 
* probability that any one individual could hi up in ‘i ie 


-,. * one group or the other. 


-@ Pretesting* --. the adninjstration Of a: procedure 
such as a test, survey, of observation, at the be- 
ginning\of a program to ascertain certain character- 
istics of treatment ‘and lls toe individuals in-. 
stablishes you base: a e 


know or do before the program begins. ‘3 { bis, 


“e Pos tatestin -- the administration of a procedure tes 
“at the end of a program to both treatment and 
comparison. participants to ascertain certain 
characteristics of participants involved in the 
program regardless of whether they bechived treat- 
ment in the program, A 


which looks like a successful result of your. program can be‘attributed 


the end.of your program, you want’ as much evidence as possible to - 
document that your, progydm, rather.than some other effect, produced 
the thange. This is what is meant" by determining internal validity. 
By applying the features of a comparjson group, randgmization, and 
pretesting to your project, you can increase the internal ater ty 
of your project. e 


~ r 


pragram will have internal validity. Then, wit this background, we 
will assist you in. Aesigning a program which re‘uces these éffects, 


of 


\ 


standardized tests for your pretesting and post-testing. . : ° 


‘Assume you have’ comp’ your program and have ote significant ° * 
changes in your participants. Did your program cause these changes 
or could your changag, nave een caused by any of the following factors? 


1: 
ve a ts ge oh py fs + + yy 8 wastes ty sf Ahead 


T. Concurrent historical events. “Someth ng “may have sacle 


outside your program which 7 influence your participants, 


Fume EXAMPLE: (Using your art appreciation ‘project... ) 
ee A-robbery at the local, museum occurred during 
ar" :. tar the time of your program and much attention é . : 
' : was given to the valuable and irreplaceable 
p ; ‘ature of the-works of art stolen. 


-, 2, Maturation, A physiological chahge takes, place with or 
, without a.specific program as a result of the normal _— 
passage of: time, 


SP at Be \** Ny 
( v 
RABIES “Most first graders Tenn’ to cut along wit ; 
lines because Of development in their: mOEEE é 
7 \ coordination during this age. 


3. Testin Participants do better or worse on a post- -test r 
merely because they have taken a pretest. : 


EXAMPLE: Your sixth grade students answer post-test ° F 
. art attitude questions much more pogitively + 64 
‘ nh because they have been sensitized t@ what 
\ : , you are trying to achieve in your program at ® 
, the time of taking the pretest attitud% 


inventory, or, ee i bet es 


4, Instrumentation. ‘The conditions under which initial ob- 
servations were made are different ¥rom the conditions 
for Final observations, 


“EXAMPLE: Your art’ attitude taventery was “inconsistent; : \ 
. ; the same student could take the same test . - 
: within some minimum time peri.u éven without a ; 
participating in any program and get radical ty 7 
different scores. ‘ oe 
EXAMPLE: You had reviewers rate the effectiveness of 7. 
units prepared by both before and after 
teachers participated in the program, but ( 
the ini®ial reviewers included several people iy 
with art history background while the final bs 
reviewers did not have comparable backgrounds. oa, 
The initial’ reviews were more critical than the ‘ 
: final reviews, 


al 


- 58 - . - 
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. ' ; ; : 7 : i . 4 . 
: 5. Regression to the mean, or statistical regression. Students ~ >, 
Fa scoring at the very high or very low ends on a test will i4 
’ ’ most probably score Closewto the test mean the second: 
e 


“time they ta us same test. 

+e, EXAMPLES For your program you took the 10% of the’ students. 
ra er i “scoring Jowest oh an art achievement test. Re-  . 
§ardless of when the students*take the achieve-- . 


” "  & ment test and regardless of what happens to them 
in an art program, lapel scores are 
x : likely to b® higher ‘e#€a second‘testing, The ° - ma 
~ explanation for this is that students scoring : oo ¥ 
, at the very high or, very low end of the@scale 
obtain’ part of their scores as a matter of luck, * | ~- 


_ : |. dn this case bad-luck. In yetaking the same test, ~- 
the luck element would change so thatgin all prob- 


GP Fey 5 ability they would nat have as much bad tuck, * ei tt 
: ~ that is they would score higher, regardless of © ad: 
#, 4 4 * "be they took ae test, again and regardless of  * 4 
( ether they participated a the. program or did: : A’. 
‘ f.% not participate.. (The opposite would happen 


to very high scorers. They had received very 5 tag 
high results as a matter of good luck and on os 
re-testing would get, on the average; lower. os ? 


* ° 
scores.) : ne 
' 6. Differences in selection of participants for comparison .« . 
and treatment groups. ~ Participants receivifig treatment 7 
. are significantly different from participants in. es 
comparison group on § gnificant variables. 
Ms EXAMPLE: “Your teacher's receiving instruction on how to mei 
, ae prepare coursésuynits were ‘selected partially f re 
ih. because they had experience in, fine arts curri~' . 
se culum development ,whereas those not partici-— / r 
: pating (those vin the compar igen group) had no P 
AV, . ° previous experience. It would not then be . 
- se rt appropriate to compare sample course units 
from the.two groups to determine if program ; ~~ : 
participants earned how to i sd units better. 
* «we ¢ 7a Experimental mortality. ~Participants differentially drop 
out of the treatment -and the comparison groups in your * 
program, — =z 
, EXAMPLE: « Your program is based on programmed instruction: 
ie or : As students successfully complete their training, 
Z they leave the program} At the end of the year, 
\ *. you post-test the students currently in the pro- . 


gram and conpare them td the same comparison group 
tite me 
. | a. 
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6 
t6, ‘ and treatment groups to. produce a change which ea ih as able to 


‘ é ir 
oh 
. » ee F 
f al - a] : i ’ a * 
P x ieee i e é i ’ Se 
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4" °° you hed in the beginning of NK program, You 
; did not have am opportunity to test all the . 
’ : ‘successful students who left the program during , 
the’ year so your: average student test score 
yas MO t show as significant a change as it ~ 
might otherwise. have. 


‘Ina til is i these seven factors which out lead to : Mihitannion 


ey 
e: 


of y ing a Migr olirk there could be interaction effects Of (two or 
more of Mthese “factors. Other factors, usually, history, maturation. 
of-testing,. interact with the selection of participants in eynpar ts 


the Progra. ~~ 


Now ‘that you are aware of tha factors which 4 may ‘invalidate evidence aie. aor 


your: program™has achi¢ved its goals, let us examine the ways in which - 
you should design your eregnan to minimize these factors. You would. 


want; so 8 ey ee, 2 2% 


| t 


e benefits of your, program and a similar group- 
hich is not receivince benefits from your prggram; and. 


nm: have one group of okittetpants/teariirs "receiving 


e to assign participants/Jearners randomly between the ©. | 
tHe groups, — 


g the Conpar son Group_and Randomization 
By having er participants in your program randomly sect ona (not . 


necessarily in al numbers) between a comparison gro@p and a group 
Which.will havg@ythe full benefit of your program, you can safeguard 
against most Mn difficulties encountered in obtaining ifternal 
validity. As & result you will be better able to show that any: 
changes occurring in your participants are a‘result of your prograin 
and not extraneous factors such as wiaturation, biased participant,- | 
selection, such as. voluntedring, and.so forth. Unfortunately, random 
assignment does fot.ctntrol for all of the internal validity factors — 
but to,design a schood pr hram which could completely control for 


each of these factors. is @eyond the scope of this neta 


bs > 


£° 
Hide do, gon obtajn eee - The key is to have your 
population, from which your participants will be determined, Peady 
before your program begins. This.may involve extensive advertising 
before your program begins or it may require soliciting the aid i 
administrators or teachers for referrals. 


ro 

; * Once yous have your population. defined, ‘randomly assign potentials ey As 

“participants (students, classes,. teachers, or whatever) to one of two ek ee 

groups--thosé who will take part in your program and those who will . i 

serve as a comparison group. Jf you will be relying on volunteers, | ; 

-» §tate during the vo Tunteering process that only a portion of those e ~* 

‘ signing up for the prograp will be able to participate; After you have - ‘ 

* =  " «your list of volunteers, randomly assign each to one.of the two QHOUPS yrs eee 

Random assignment is usua]ly done in one of,several ways: flipping © ... . {° 

a coin to see if a participant -is in the treatment or comparison group; ar 

ee » usin a random numbers chart to select.two groups from a larger. gréup;. ° 

» —, drawing names from a hat to place people in treatment or comparison ; 
‘is "groups. Once you"have your,two groups, be sure to retain the list of 

: peopte in your comparison group for your ‘post- ~test purposes. .. ‘ 


ce lt The one pons tderation you must. always keep ‘ta mind oe your A, mig 


. ‘two groups is: that they must come from exactly the same population. ' uf 
Dividing a'class into those who valunteer to attend your program and oe 
calling the others in the class -your comparison group is not random ,* wel! 


| _ selection. _ Random assignment involves a process like puttirig all names 4 
’ jn a.hat and sie drawing out one name at’a time until mle have the 
fans : number, requi your program, The remaining: names would be ye 
ee qtoup. ; ee res Obs 
* e e : : : zi » 
What db you do with: a compart group? - The comparison “broup, and at *. 
your program participants should have exactly the same experiences ~ a : 
except that your program.participants are exposed to your program and 
® ‘the, comparison group members are not. : | ‘Nsehe F * eye : 


‘In more exotic program ‘designs the conparizen group might receive : x 
- a pavattel treatment under the same gondition that the program partici- — So ae 
% pants experiéice, For example, if you pull a sample of elementary i 
" students from class every Friday to receive an hour's worth of art -” of, 
appreciation instruction, You. would at the same time remove the compari = ee aes 
son students from the regular class and give then an hour's worth of |. . s ge 
* . instruction in a totally unrelated ar raja sence or remadial’>, i.e: °° 
1 reading. If both treatment and comparison, groups are pulled from thetr. a 
regular classes, they are being tréated more eqhally than if aust treat- 
ment individuals are removed from thé regula class. 
: . ed te, ee re ant mn 
+.) st (ia WH A = . ‘ ~ 
To Pretest or Not-To Pretest : oo 


If you have, indéed ,« assigned students randomly ‘to the treatment on 
the comparison group, you should probably not pretest. Giving students 
a pretest may’ ‘influence their response to the program, that ‘is, the ; 
pretest would become part of the learning experience. It would then 

be difficult to generalize the results of the program to populations 
which may later participate in the program but may not receive pretests. 
For example, if you wanted to test whether use of your newly-designed 
curriculum unit hélped to ‘reduce stereotyped led you would 


* 
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42 pean we ak . 
. ee, “gt Yan to’ Pretest’ youd Su Subjects: 0 seantyiad wtittides ” 
ig Steg “be ore sing: them to<the“units. .They.wauld likely become. see 
a 4 “+ té the anbs ‘yOu were going-to.teach them and ‘the. connie pup 


“\ whether ‘the urfi shad an effect on. your part pants Tf, you fo a st, oe 


Mie however’: your Fests pretest af. a re ular. ae oF ‘the general «4 ae 
educa ional program, use. of the pretest wou can not adversely a 
dat fect your program,*. 1A, yfact,*you would wy 0 compare ‘pretest scores * 
f treat ent and.-coi apSPison groups to -be > ie t ir average scores , 
ere the same and “hence that the .two grpups are €0 ihe nother * i 
“Convenience of pretest is that those resylts- together with post-test e'8 
.. régui.ts.'allpws. the amalyst to use*more genes statistical procedures - « Otay: * 
: * yin: data anatysis. ..We.assume, of course, that if Brevests are\used,. 
$0 ‘the: appropria Jets Will be chosen and that. they: will be administered 
Rsk, correctly,» eee Mrs ‘ eS 
eRe = ae : car “ m ¢ " 7 ose ny : Se > a 5: 


ord aot tetescng (' aK ig © Ft any j : 
- A Wor re | a 2 ee ae _ Ks 


ee > te Bis “ike procedures sou e to determing whether voli program has jad an. 
+ aa Fr ‘effect, on your“participants do not have to be. paper and penct) tests. 
7 ae Observations, ‘interview,procedures, or’ school records, such as attendance 
bao - data, are some of thé alternatives to standard testing procedures,  . 
oat . _ Whatever, the method(s) necessary for your program, r ber that’ both: =... ye: 


\ 


° your participants and the ‘€omparisor aro must. eget -tested by ie 
4. interview, ' ‘observation, or whatever. » , e 3 i 
a, a) : 


é 


_. | There shout be’absolutely no distinction made’ between taking final’ 
gr* data from the participants and taking final data from the.conparison: 
“ _ group.. For exampley if. you individually interview at thetend of the: | 
: program, you should use,the same interviewers for individuals from 
a both treatment\and comparison grodps. . In fact, it is better if the 
[ _. interviewers do\not. know wheter vey are interviewing a treatment. OF ae 
eh am Nest dada person, ou eae fs 


ae ; 8 col rs you' do“f darticipants drop out of" you \g program? First, 

Th oo * you’ W ant td ascertain there is something systematic about the: drop-  ' 
crit yore outs,» Did. they’ di like eer Did.they "graduate?" .1f you have - 
th ie % pretest. sae ‘available fhis might be useful in identifying any trends. 


ie vySetondly, ou should probably Mclude dropotits in your ‘end of ‘program 
ay ‘y | post-testing even though this would proBably give you a conservative 
3a; ike id of ‘he hee of ‘your) program. 4 
- “~ . A x E: | : t « 
Le me : 
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 EXTERIAL VALIDITY: aa s 38" f ek 
eo 


." your program fs accomplishing what it proposes to Ae that you are able 
a 


: or district. However, you’want to have your progfaih also make a greater 7 


- validity. How generafizablé 


‘ : ae é : * w I~ 4 
; af :< % i 
.. q 


“Let * suppose) that you have ded your SrOotan eine the techniques 
‘of randomization, comparison groups,, and appropriate testing situation. . | 
Let -us further} suppose that you are able to determine whether or not ~~~ 7% 
to' tel] that your program Solves the ‘problem particular to your school 
contribution to educational advancement than just solving a local’ educa=. 
~tional-problem, You want to. know’ ff your prdgram would be of. valye to 
others.’ Are there other pTacés® which have the same problem your program 
is designed to solve? ‘Would your’ program Premiee re same outcomes , ~ 
_ elsewhere?, 8 4. ‘n Ne ie 
“The questions you are sskine pow relate be the eeue of Gee Ng 
Ss your program? If the program works dor we oa 
Rist will it work for gthers?. There are.ng formulae for ensuring that , het : 
our prograni is’ externally valid but there are factors te consider. ‘in 
Pieterninites ever ae your program would be we elsewhere. 


“If we assume that ygur program 1s) eeseondtna 9 a need that is got ime 
unique .to-your situation but that could be found in many other @duca-. 
tional situations, the next consideration ‘is whether these other educa-, 
Xt nal settings would have the resources’.and the enviroment necessary ,, 
or ‘conducting a simi Var. evil You: would, therefore, have to review 
he human. resources and material resources to: sée .if other. settings could. 
* supply similar resources | and you would want to see whether the climate » 
under which you would-begin’® ands conduct. your, program is .dependent on 

in political.factors which.are Likely to be reproducible. You, 
might: want, to ask some of. the: follawing questions to stimulate your” ; 

“ Btaking: about how pplicabler yop peesrans ‘would be in another. setting. 


e Are my resources jayafJable elsewhere? = we 4 “ Lass 
F » ie 

’ e Whatsare the’ particutar fenkites of my ari : 
* population?¥ Is. the program aimed at a specific 

age level? achievement level? family background, 

/ grouping? How many Jearners cart/must* my program 


+ ¢ 
', ( accomodate? A pe. oak 


_@ What are the particular feature of my staff? 
Must my instructors have certaih qualifi#ations, 
training of experiénce? What types: of Support 
staff are needed to eompléhent my program in- 


structors?, Will my program rely on specialized 


consultants? } a 5a 
«< *~ ; 
a we r - . ro 
* e ta, 
\ oo 4 \ 
¢ 'v be 7 
) nt ¥ 55 a) is 
' bot il 
] 
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“! bei 


e. 2 ; a 
e What. i of “eauel et and ‘vatertaty are necessary F yp 
for the program? Will I have to develop specific. el { 
. training materials? Will the- program depend on . 
having certain commercial instructional materials? 
What types. of phystgal’ facilities will be program _& 
_ require? ! #4 


e Is a certain type of climate necessay, for the : a 
ra operation of the program? What type ‘of political °° ’ 
support is needled to haves) the program adopted? —s _— 


In answering these and ratated questions about the needs of your program, 

think ‘of othe settings -you are familiar with in Massachusetts and in . 

the country at large. WouJg ‘these other settings have similar resources | a 
and climates? The more settings that have these resources; and climates, ; 
the more generalizable -is your program. ‘That is, the greater its \ 

external validity. 


Theré 1§ one other’ aspect to ‘fa issue of external’ validity for your 
program and that has to do ‘with thé experimental nature of your new 
progra Essentially this means that for your program to have external 


an.experiment. For example, would your program have the same : ‘4 
outcomes for population’ where no pretest (which might sensitize parti- 
“cipants)' is administered? Would participants respond to a program which 


es success of your program cannot be dependent on your program '* . - 


Vgsponded to an innovative’ program? 


og has been in the syste for.several yeas in the same manner as they 
t 


rae : ; / 
Th isdue of. setureel validity is important specially \irt developing a 


able ‘Tit IV program because you want to make a contkibution not only ere ee 
to ygur local s¥stem but to other educational systems. The more:your 
‘program addresses general. educational needs sand the more care* y “use 
in developing your program so that it is feasiblé for other systems to ~ 
adopt or adapt,’ the greater the likelihood that your Title 1V inno ative 
program wil] have a asian ygern impact on education. 
—_ 
j- “SUMMARY ‘ : re ad 
$ ey r 
To establish a good ‘program it is not necessary that you become a 
full-fledged evaluator, but it is important that you becone familiar : i 
., with some of the elements that go, into designing a program that can : 
_ document for itself its degree of success. This booklet serves to give 
you the basis for establishingyjust suchja program by highlighting the. . mo 
two major areas: By cessery for developing \sound programs. * The areas are:, * A, 
\ e developing appropriate. progr m objectives = 
e conducting your program withim an / 
™ experimental framework 
. 


o 4f 
yf 


Only through developing a sound’ program can you be sure that your Title 


‘IV program has both internal and external validity; that is, that your 


program accomplishes what ,it says it is supposed to be ‘aotng and that 
it is applicable to other ; Situations. 


That | our program have Fae validity, that it dotinent its success 5, 


. 


is. important for any program t fo help you obtain internal validity 
for your, Title IV program, we suggest. that you model your program with 

a treatment group and with a comparison group and that you assign ‘parti-. 
cipants by chance (randomly) between the two groups. Before the pro- . 
gram begins you-coblect all necessary data from both groups: to demonstrate, 
the degree to which they possess the characteristics that you will be. 


wantinggto change through participation in the program as long as the 


process of collecting the data does not form any sort of teaching of " 


the participants and as long as it does not sensitize the POLAT EI PAIES 
to what you expect to do with your project. - 


‘During the. course of your program, people in the comparison and bea 


ment groups should be treated exactly the same and have all of tf® same 
experiences except that your treatment group participates in your pro- 
gram and the comparison group does not. Other ‘than program ere 
your two groups a 


At the end of your program, you should test all canis in your treatment 
group and in-the comparison group for all ‘of ,the changes that you expect 


‘your program to froduce in those’ who participated in it.‘ With this 


basic program design you will be abl® to document to a better degree if 


“!your program is producing- the changes you want it to prodice. » 


That your program have external validity, that it be generalizable, is 
especially important to Title IV projects. In this booklet are sug- 
gested topics to consider when viewing the educational problems your 
project is attempting to solve and the‘Means. you-will use in solving 
these problems. The basic questions: to ask er external. validity are: 


e Is your project addressing a common educational 
problem? =. \ 


_@ Mould others be able to do the same sorts of 
things that you will be doing in your project? 


One final reminder as you begin the process - Of implementing: your project 
is to urge you to seek help.from those specializing in project evaluation, 
There obviously is more to developing a project than what is contained 

in this booklet or even in what is contained in tha references noted 
throughout. .There are also times, although a lot Yess frequently than 
what you will want to thnk, when the model we hav described is inappro- 
priate to use for a project. For these and other-reasons the evaluation 
specialist can be of ‘great assistance to you in ‘formulating and verifying 
a successful project, ; 


‘ “se 
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“FOCUS ON VALIDATION 


oar a. 


| - THE: STRUCTURE 


While much has been written and discussed aboist the validation process, 
specific and practical validation materials are not easy to come by, 
Merrimack Education Center has produced a validation process which was 
tested ‘using the- Individually. Guided Education propran. MEC's validatTén 
products are reproduced here since they were a p of the materials 
reviewed by the project directors. This:chapter carries in detail the 
results of a validation process undertaken. in the Chelmsford Public 
Schools from the fall of 1971 to thé end of September 1973, The next 
chapter is based upon a simulated validation process. / 


This chapter, and. the’ preceding one, review two key Pisheiits: in the 
useful development of innovative programs: evaluation and validation, 
Tradition and familiarity are the twins that seemingly assure the " 
survival of "regular" school progfams, New Programs, and ‘ideas can : 
rarely expect. acceptance and @ degree of success“unless. they in.turn | 
. adopt evaluation and idation as their twins.’ Any. new; program expected 
to last longer\than a’season or beyond the end OF fort money must. gird 
“itself for the inevitable contest. / 
We begin with a biter: ‘project summary, fi Towne by sections on program 
outcomes, effectiveness, exportability,’ and budget requirements ,, 


' { } , / Vv ty * mh 


PROJECT SUMMARY - 2% : ae iS “ 


ue \ . ye F ; i , , 
It will be useful for you to have an overview of the project upon which 


' . this validation:process was based. Thu®, the following summary of the 7 


project is offered. ©: . 


Individually Guided Education (IGE) may be defined as an inservice pro- 
gram for elementary school personnel that leads to individualized 
instruction. It is, Jong term jn nature, requiring three t&five years 
for & specific league to accomplish 4 majority of the outcomes. bs 


° 
The total IGE inservice program centers around four components: 


@ The Multiunit Organization -- This plan couples 
a team teaching approach, utilizing team leaders, 

“ with’ pupil assignment on a multiage grouped basis. 
This Fonad oy making structure allows many 
decisions. to be made by the Instructional Improve- 
ment Committee, a group composed of the unit leaders 
and the principal. 


ii 


inne 


i: 
. % 
Sey i 
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a ,e The Learning Program -- This component is. the Sy 

‘ii ; heart: of IGE. Subcomponents include the diag- 
nostic prescriptive approach, use of specific ... 
objectives , , assessment, Hultinnedia. materials, 
“emphasis placed upon pupil selection of 

. materials and objectives, fitting: the. activity db 
' to the learning style of the pupil, and allow um 
ae ’ es Ning the learner to proceed at his own rate, 


e Home-Schoo! ‘Communication -- Emphasis is-given. 

' to: developing a ‘planned program of community : 
involvement. This component includes not only 
explaining the program to the parents but also 
encouraging them to participate in it as 
volunteers and aides. 


@ League Linkages -- Individually guided Edaca- 
tion is.implemented in several schools ina 
geographic region with an, intermediate agency 
as the facilitating agent. Linking of schools 
in this manner provides a source of peer. —— ' 

: ‘support, an exchange’ of ideas, and a ‘temporary 
é‘ social system that cAn enhance the implementa- -’ . 
, tion: The intermediate agency gives legitimacy 
to the implementatio aind provides the coar- - 
dinating element.’ , \y 


x 
. \ 


MAJOR. PROGRAM OUTCOMES ae 


a) 


to he PS, the major BronKent objéctives upon which your neotect is based. 
' These objectives should reflect any formal! ennOges that have been made 
Since. the submission of the original ‘di ous 
& ° 
Project Objectives ! a ee é ; 


1. Prdgram Area: Staff Development ° 


Leadership training for principals and unit leaders of the 
league is’ a comprehensive ongoing process Of renewal. Indeed, 
Individually Guided Education is defined as an inservice program for 
elementary school personne] ‘that leads to individualized instruction . 
for students. 


: Objective One ---To provide implementation training programs, 
4 using appropridte training materials for administratjve and instructional 
personnel of the League schools, as well] as parentag) in order to: 


@ Build group identification ‘ 


- 67 - 


- $ ey 3 Soy “ * . * 
e A J f 
. 
’ 


Develop staff cprmi tment to innovative behavior i 


Help staff become familiar with IGE materials, 
objectives, and strategies 


* 


Facilitate parent understanding and support for . 


IGE 5 - = ? ‘ 5 
> : ; 
ees fe - Enabling Objectives: “, a ‘ 
‘e @ To identify and develop tens capabilities 
and potential in Mieka unit Bae la. and 
administrators - ' 
a \ 


e To select and/or develop training programs for - 
League schools targeted to the following y 
populations--principals, superintendents, i) 
teachers, unit leaders, and paraprofessionals 


e Ta evaluate the IGE materials and strategies 


, ‘To build information exchange clannels at variou® levels in- 
cluding external and ae dissentination. 


Objective Two += To establish a veaoad of Schools" characbare 
ized by,a coordinated: program of interaction and training. A league © ‘ 
should provide peer support, a, communication network, research assistance, . 
and service support. . League information, should be- disseminated to 
other regional, schools. 


Enab1 ing, notectivess pO * 
f e Jo increase parent participation and support for’ 
IGE OnIEE TIVES ‘ 
e To maintain. ace update an educational clearing- i ~ 


house On resource people, successful practices, 
and information 


@ To provide tréMhed individuals from a League S 
. agency who will sérVe as field agents . 
at 
e “To disseminate information. to League schools | 


‘and othef target audiences - z 
» 
; SR Stat %, AE 
Bg ~ 
7 2 td ’ 


. s. Program Area: Support Services. aia. 
Support services for marshalling internal, and external resources — 


to meet identified needs’. Needs are identified and resources brokered 
to meet those’ needs. 


Objective Three -- To assist teachers rh developing the skills 
“related to the IGE Learning Cycle (assessmenty general and specific 
goal setting, planning diversified learning experiences, reassessment, 
and recycling); and to assist teachers in implementing an IGE instruc- 
* tional program, 
o . . A 


¥ 
Enabling Objectives: 


e To identify needed support systems and services 
in implementation of the 35 IGE outcomes 


e To target specific resources according to. the 


: successful practices, human resources, and 
7, ‘ targeted information products . 
e 


_ To establish effective linkages with external 

' programs and resources. (Colleges and Universi- . 
ties, I/D/E/A-Kettering, GCE SRO IRES EONS 

and state Department) — 
Program Are : Evaludtion 


’ 


¢ « 
. League schools. 
. 4 


Enabling dbdentivees 


e To provide three to five Peer Process evaluatibns 
in selected schools 
eS . e To maintain anecdotal records dM league activities 


== e To develop the muftiunit concept in each of the 
h league schools by assisting league leaders to 


develop skill and confidence in school-wide ) .) OE 


decision making; assisting league teachers to 
develop the ability to use the Unit approdch in 
. organizing the instructional program; to provide 
o°.% consultant analytical services to league schools § 
r from various agencies 
a 


needs identified. These will be in the form of as 


Monitor, assess, and evaluate tip ienentation of IGE in Project 


THE. EFFECTIVENESS OF THE ACTIVITIES 


The League Process identifies and develops leadership ‘capabilities and 4 


potential “in teachers, unit leaders, and administrators. Training 
, programs for League. schools are developed and targeted to- the re NO 
‘ populations: ‘ 


e@ Principals 

_) Superintendents 
@ Teachers 
e@ Unit Leaders 


e '.Paraprofessionals 
Project League provides implementation training programs, using appro- 
priate I/D/E/A materials for administrative and instructional npereanne) 
as well aSgparents, in order to: 
P P 
@. Build group identification 


> 


e Develop staff commitment to innovative we 


e Help staff ‘become familiar with IGE materials, 
objectives, and strategies 


® Facilitate: parent understanding and support for 
7 IGE 

A regularly scheduled series of workshops, conferences; and educational 
‘exhibits is established by the League. These consist of HUB meetings, ~ 


courses on Wiseonsin Reading, SCIS, and the like, principal-unit leader’ 
workshops, and prograns for teachers. Tablo’ indicates a comprehensive 


scheduling of these, activities. 


“PROJECT LEAGUE MEETING INFORMATION 


TABLE 6 


TOPIC DATE ~ PLACE NUMBER AND TYPE OF PARTICIPANTS, 


i ‘| dahawiarad bbdaetivas 


P/UL Workshop on 35 Outcomes 


Individualjzed Instruction 
Materials Exhibit and Guest 
Speaker, on Management Systems 


‘| Massachusetts Association: for 
School Principals "IGE" -- 
Eiaine McGregor 


} 
IGE Superintendents and Principals 
"Problems, Needs, Concerns, 
Resources” 


P/UL waricho’s on "Problems, 
Needs, Concerns" | 


bs : 


IGE Superintendents and“ Principals 
Sharing Successful Practices. 


Principals' Meetings } : 
Unit Leader QOnsultants from 
Wisconsin and. Coloradq, 


’ 


Monthly HUB Meetings 


* 9/14; 


9/09/71 


12/13/71. 4 
11/02/71 


10/13/71 


12/09/71 


~ 


11/27/71 


1/26/72 


“Monthly” 


10/05; 
11/09; 12/07; 
Wits syigs 
4/04 


Billerica 


Andover 


Westford 


Amherst, 


Andover 


’ Andover 


“HEC oFFtee 


ey. ee 


Lowel] 


® . 


School 
Buildings 


60 Principals, Tea abhers/ 4 Unit 
Leaders ~ « 


60 Principals and ai Leaders » 


125 Principals, Teachebs, and Unit 
Leaders (Dinner. Speaker = 45°Persons) 


\ 


8 Srigeinats. 
, 4 


. 
20 Superintendents and Principals 
& 


50 Principals and Unit Leaders 


Superintendents and Principals 


/) 


All Staff Members — 


13 Teachers and“ Unit Leaders 


Table a. | ar “4 i | . ® i 
_ Project Leagué Meeting Information : 
* (eonsinied) : 5 


: 3 ‘ ‘= . : shih he id 
HUB Ad Hoc Committee Progress 6 meetings . MEC Office 8 Teachers and Unit Leaders 
Reporting Sy>cems es ' 
IMC Specialists.Operating the 3-meetings | ‘ S$chools - * 10 Specialists ilarariane: Ruste * 
IMC ” A - ; visualsPersonnel ,, etc.) 
Parent Advisory Conmittee | 3 meetings | | MEC Office ake Parents 
Informational Needs " ; 
Assessment Home-School : 3 i 
Communigations 
OTHER LEAGUE ACTIVITIES . 
All League Educational Fair == | 5/31/72, _ School and a. i 


16 Big Ideas Selected by ~ . h getting Green 
Participating Scfhisels a 


Summer Course - Graduate Course , 
for Unit Leaders (/In Conjunction Y . oo 


with Fitchburg-s ate College) 


Principals’ Meeting ~ IPI | 5/25/72 '.  MEC Office 


» = a 


? - 72 - 


: ne 
Fab wiiaes 


s * 
A ea! a 


KS ois 
~ Chronolog--Conférences, Workshops; Mee trings omer : 


ceased) 


| September 12/72 
September ‘26 

| October 1. 
October 3 

October - 

“| Octdber 

October § 
October 10 
October 12 
Ogtober 16 

“4 0c ber aT on 

mS October 37,°18;. ‘19 


dhe 19°. 
October 

Qctober 

October : 

October . 
Yovember 8,95. 
“Noveinber on 


Month] y. Meetigg: 


“Monthly, Meeting 
Planning. Meeting ° 
“MESPA :, ee 
Monthly Meeting. ‘ 
rPeer Process’ 
Wisconsin Study Skills 
Cohasset * 
Peer Process 
Monthly Meeting 
Learning Styles 
"Wisconsin Facilitator 


: Training 


Mass’. ACES ’ 
‘Monthly. Meeting 3 
Peér Process ; 

Fitchburg, 

* Al Cullen 
’ Principals’ Workstiop 
“Monthty Hegt ing 


Parent Advisory » 


» Comni ttee ; 
deer brocdss *§ 


‘Continuation Grant” - 


Negstetter a 


wh 
wy ' 


or. Glasetic’: “' 
1972-73 Planning 
Plan Visits to McKay 


- 


IGE Dissemination 
Mckay °° a 


. Plannip ‘ fommi ttee 


rite: Tie. 
News]etter 


- 


4 


Principals — 
News tetter Committee 
Libby O'Connor, John Allen . 
nichts aie 


- 


HUB‘ Members. 


| 


~McKay Teachers 
Principals) 


Principals 
Shattuck Street School 


v: 


} 


Newsletter Comittee 
Mckay apr iocioa s 


feachers, Unit Lgaders 


“» -Chronolog 


“DATE ‘| PURPOSE | 


ao) 


a 
. 


Table 6 


November 18 
November 20 
November 21 
November 27 
December 5 - 
vbécember 6 

December: 11 


December 13 


Decenber* 14 
1/73 
9. 
January 10 
January 11 


January 
January 


Januéry 1&, | 


January 29, 
“| January 


February 1,52, 


February 6 


February 6, 


| Februapy 7 
February A 2 
February’13 
February 14 


re 


A‘. 


(Continued) 


} 


Learning Styles Unit Meeting ~ 3 
Monthly Meeting Unit Meeting 
IGE Dissemination eo an 
Monthly Meeting - Newsletter * 
Monthly Meeting "What's. Happening'® 
Planni IGE National Convention* 
“Monthly Meeting "Learning Modes" 
pure ae a ce 
Monthly Meeting Newsléttery 
IGE Overview ae : 
Monthly Meeting » League Evaluation Design 
Monthly Meeting Newsletter > a 
Behavioral Objectives Storrow School Ay. * 
‘| Monthly Heeting Individually Guided “Motiyation 
-| Unit Leadeg @rainings , A 
League Evaluation \, \ 
Frincipals/Unit . ' 
leacers * 
| Monthly Meeting” - Reading Skil! Packages 
Atlanta Conference Pgs 
Nénthly Meeting™*’ Newsletter 
Nonthly Meet ing | 106 Functioning tee 
; IGE Overview 
_{Paraprofessional i - 
Workshop 


5 


& 


. |; Vaughn \, sa 


Principals 
Principals ; 
Lawrence (Primary Sghool) 
Newsletter Committee -~ .“ 
HUB Members 


Principals# 
ae’ 


Newsletter Committee _ 
Methuen Teachers . 1 
HUB Members ‘ 
-Newsletter Committee 


Principals . ° ¥&° 


/* 


Vaughn | , 


5 pet @ 
‘HUB Members, 


° # 


* 


Newsletter Commi ttee 
Principals pp. 


\ 


A tate role has. been, acide’: to League srhactials as iss shown in| . 
Table 7 -- Project League Principals' Meeting Agenda.’ This table indi- 
cates that the meetings are designed ta assist principals in their 
leadership functions, provide for exchange of ideas and decision making; 


contribute to self-renewal processes of both the principals (as ~ o 


individuals) and school building IGE programs. 


= 


. : TABLE 7 : : So a? 
PROJECT LEAGUE PRINCIPALS' MEETING AGENDA TOPICS ‘ 


ay 4 


AGENDA TOPICS 


Ealatatton and discussion of ERIC Information System. 
Administrative details. 

. MESPA program at Amherst — 
IGE monitoring by principal and. by MEC 

, Review of preschool workshops 

IGE. implementation problems 


“a. When more than one curricuTum area 1s bakde 
individual ized 7, 


b, When, within. one schdol, units dre individual izitig 
‘different instructional areas \ < 
‘\ 


s 


MEC Staff: Reflections oe observations 


Rote of prfncipal in indi vidual ing instruction: 
Dr. ‘Robert Anderson } 


. Social hour, dinner * . . \ 4 
League principals’ IGE quotient to date \ 
Planning nd programming for IGE implementation 


“s 


Plan. fon. cont \nued ‘inp Lemertyation\ of IGE. 

Me 
se ahead to eptember 1972. | \ 
att Collect building implementation matrices | 


| AGENDA, TOPICS ™ 


1/26/72 


(Principals and Superintendents Joint Meeting). 
1. Lunch : 
2. Future implementation strategies 
3. Perceptions of the present; alternatives for the future 


2/17/72: | (1. Peer evaluation process 


a. General information _ 

b. Précedures . 

c. Developmegt ‘of evaluative criteria. 
d. Implementation steps 


Title III feedback 


3/23/72 : 
. 2. Peer evaluation report 


3. Inservice materials | 


> 
. Administrative’ announcements 


» ae Budgets é' ‘ he 
- I/D/E/A evaluation $ Pes 

‘ c. ‘IGE contract ’ : ; 

d. Wisconsin, study skills ‘ 

e. NEPTE proposal five y, 


o 


& 

2. Independent aydit 
ay Grouping ang 

4. Other’ } 


Evening meeting Wi 
\ . Laer a 9 


Gocialemeeting planned. for league rincipals & facilitator 
ae EY F . ‘ ’ ‘ 


[om 
ry 


“th 


Table 7 (continued) f 


_, AGENDA TOPICS 


ah fe s 
Continuation grant report 


a. Field agent plan ~ 

b. HUB committee 

c. Peerreyaluation : . 
. Budgst \ag locations 4 


. Planning committee, 1972-73} 

. Inventory of inservice materials 
IGE curriculum areas 
. Preschool oN. 


a 


Schedule “fortfuture meetings 


Use of audio-visual aos -- "Unit Meeting" 
Guidelines for visitation |\ P 
Storrow School program report 
4*NDEA Title III proposal Jever pant 


5. Peer process 


v 


"Learning Modes" 
Involvement of teachers - _ ‘ 
Harrington School progress report 
Social hour, dinner 


"Parent Information Center" 
‘"Communi@ation with Parents" -- filmstrip | 

"Unit Leaders Training" ..° aS 
: Woburn Street School prouress report 


iz "Many Roads" 
2. TIC functioning” ‘ 
+3. Col. Robinson Schoo’ progress report : 


2/12/73 


es ss | © , 4 j 
IGE implementation guidelines suggest that each League organ: a HUB ; 
Conmittee, representing teachers and unit leaders, to devel@p communica- 
tion and sharing of resources. Project League's HUB Committée consists 


of representatives from each school, working with the League Facilitator, 
t 


meeting monthly. , Table 8 shows the HUB Commit 


9/14/71 


> 


ee agenda topics. x 


. ; \ 
E 8 . a @ 


4  TABLI 
“4 PROJECT LEAGUE Se ee TOPICS 


Ts. 
2. 

3. 
4, 


5. 


ce: 


12/7/71 


VWTs72 


1. 


2. 


3 


5. 


Ts 


. Sharing of. successes 


"| 4° Plan for League) Project 


Other ge | ? 


Rlans for all League seins scheduled for ® 
October 26, 1971 ‘ ‘ ol 


WESPA meeting at Amherst 
Newsletter ~ 
Information exchange > ee ce 


a. Identification of successful. practices P 
b. Resource péople . « 
c. Problem areas = \ “>: 


Plans for next meeting ‘ . s ° , 
| 


Announcements 
Problem sharing 


1972 Baaautannas 
Principal 's/Superintendent 's ests, 1/27/72 a: 
Ad hoc Gommittee on reporting pupil progress ’ | }. 
Other : 
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Table. 8 (Continued) 


-AGENDA TOPICS 
\ ety - ; : . 
Review of material for reporting 
« 2. Test bank ad hoc committee as 
, "3. Specialists unit . 
Announcements f, : 3 pts 
Test bank report ~~: ( 
eM . Comittee reports | . 
All league meeting 
Other 
wo 
. ct 
. re Announcements . 
’ ‘Planning all league conferences 
Other . ; 
- ~K 
' { , | 
Pad ° 
« 
a $ t 
\ 
N ; ” ‘ 
t - wu : < P \ P 
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gs were planned and five were held. The Pa 

tively prepared and HUB representatives were 

'fromtheir respective I1Cs.° In addition to 

ing of two all league meetings, the HUB © ; wy 
Committee organized two ad hoc committees. HUB Committee activities 
were given very positive ratings by both principals and unit leaders < 

as they evaluated League operations. 


Five HUB Commiftee meeti 
agenda items Were cooper 
encouraged td seek input 
contributing/ to the plan 


TABLE a 
PROJECT LEAGUE. HUB COMMITTEE “NEETING AGENDA TOPICS 
Fd . 


| DATE | | AGENDA TOPICS oe ees 


10/3/72 |. 1. 1972-73 organization and planning * Sak | 
2. Monthly newsletter : ; e - “5% 


Exchange of news items 


-2. Sharing of ‘successful instructiona materials in 
IGE prograns 


Planning for 1973 
Zs Small group discussion--"What' 's Happening?" 
‘Newsletter 


1. Review and disc sion of League evaluation design 


2. Mini-grants oo ee 
3. Report from superintendent ‘conference plannitg 
, committee 4 : . ae 
2/6/73 ‘ ie Reading skiy1: packages ° : : ; ce ow vet - 
Ze Winslow School progress report: ws ; 
: 3 7 
: - 
. _—v * ‘ [ , | 
! f 
ry Y ‘ 
e | % , ¥ 


a 
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One abjective was to provide sufficient information. to the public so that 
- the IGE/MUS-E innovation was seen as acceptable to the community. A 
‘parent. information center with appropriate mateirals has been developed — *. 
** and installed in several of the schools. By June all of the schools had 
‘done. so. Project staff members made themselves available to answer 
inquiries from outside the project as well as plan. suitable activities to ' 

- disseminate IGE/MUS-E within the region. Monthly newsletter, IGE/MUS-E 
micropaks, workshops; HUB meetings, resource exchanges, and ERIC services - 
have.all been provided in an effort to Support the dissemination of ' 
IGE/MUS-E.*' \ ‘ 


Innovative programs ‘depend upon. many groups ‘for. vival and success. 
Parents and citizen- -taxpayers need to understand in order to support. 
_ + Through Parent Advisory Counc4l meetings, both by individual school 
Pint building, and on a League basis, some of this-communication-is provided. ‘ 

Printed publications through newsletter format and news neleases to local . 
papers are also indicated here. ‘Teacher aides, often_community repre- 
rearaas b are advocates of the schools. Additionally, the League has. 
instituted a series of meetings in each.individudT building led by a 
member of a Parent Advisory Committee for the League, a parent in an 
IGE school, / hag? been instrumental in assisting each school to set — 
up a parent i mation center to display attractive brochures and 

; visuals: that have, been designed for this purpose. ce - 


—w 


Informatidn regarding Parent Advisory Conmittee iiestings: | can be found : 

in Table 6 in adel ston. to the monthly publication of. the League Newsletter. ., . ! 

t On-site visitations ri — made to each building by. iia Leagte es 
facilitator.’ Six of tHe League schools have been’ visited by the Linker 
‘field agent; individual. bué#lding workshops shave been held on Learning, 
Styles or the Wisconsin Reading Design on four separate occasions, 
Leaders of these latter workshops were. MEC staff.members., — A 


_ MEC staff is prepared to fill microfiche and hardcopy orders for docu- 
“ments ona daily basis. Each school has an information service sub- 
scription,and can call and order documents listed’ in the League News- 
letter or in” their montWly .issues of Resources’ in Education.: Table 5 

, ‘ indicates an exemplary listing of materials that has been areenininees 
ae * yy. “4 + to 0GE. sch OTs as part of their ERIC subscriptian service, z 


’ _ League dissemination activities have been ‘expanded. to include overview 
**% "conferences and information conferences and workshops on several 
eaeeRe NONE: “(See AED J FMeOTIR Ploy, PIPER NENT ONS f: ark Nie 
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” *IGE/ == in the multi-unit school - elementary : ee iP ot oe 
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TABLE 10 ; 
' INFORMATION DISSEMINATED 


1/D/E/A Learning Styles 


1/D/E/A Assessment Booklet 


"Specialists in IGE” (Filmstrip) 


List of Local Resource Peoplé (IGM) 


Assessment Bibliography 


IGE Newsletter 


ALERT Summaries (IGE, IMS) 


Wisconsin Curriculum Serjes 


Learning Styles Booklet | 
List of Comgéiter-generated Searches 


. ‘ 


é Project,Leaque Ipventory of Materials 


Ps 


ED 148 - Educational Psychology. €curse Outline A 


Staff Development Brochures 


Parent: Information Brdchures 


Home and School Bibliography 


Educational Information Shop 


be : . , 
Individual ized Instruction Bibliography 


rh 
PREP Summary — ~ 
"Education U.S.A. Summary gg oat we 


" Education Recaps oe Uae of 


Individualized Instruction, CEDAR. Report 


"Can Schools | Up-date vere Approach to Education?" 


“a's 


States ‘implementing IGE. 


. : 2 


- An annual’ ‘needs xésestnetit is adninistered in each of the schools; a- 


survey questionnaire’ designed to\ determine the familiarity of .the pro- ; 
fessional staff members on*al1. educational concepts is.analyzed. The - , ar 


“survey has fevealed strong interest in successful practices and curricu- | 


lum guides. Attempts have been initiated to share successful. practices: ae 


. through the HUB exchange and through principals meetings. Workshops in- 


Individually Guided Motivation and maGI peng pupils, for fesrang have 


been held this year. Maes ve 7 | are 
a > en. 3 es 
Once particular needs of a given ‘school .are: ‘identified, activities are © \ 


set inta motion to resolve the particular problems.. §chool visitations 
by the League Facilitator have been instrumental. in this concerit ins 
addition to the scheduled conferences, workshops, and meetings. On-site 
visitations, "information exchdnge networks, and staff development pro- 


“ grams are seén as the primary activities and procedures for this objective. 
- The-League Facilitator has participated in®a National IGE- Conference 


which examined these same issues on the broader anole of schools -in many + 


Important tools for dteseitnating’ ' information about IGE are the handbook,” 
Implementation Guide, filmstrip sets, and films. Thése materials are to. 
be used in a variety of ways during P/UL and staff workshops. The .- 


‘ materials are designed to transmit information, develop skills, stimulate’ 


discussions, help change attitudés,° and minimize misperceptions. The 
materials also are to play an-important role in -the self-improvement 
process. Since it is assumed that IGE will not be implemented easily, 

the materials are also designed to be used in a continuous se] f- improvement 
process.. Table 1] shows teachers' and unit leaders‘ responses to the.» , 
question, "What materials haye been helpful in preparing you for IGE?" 


“During ‘the first-year of Project League, the Marsh School volunteered to 


undergo: a Peer Precess evaluation bya team of principals. In the. hou 
second year of IGE implementation, the McKay: Campus: School, and the ‘ 
Robinson School. had already participated in the feer Process: evaluation. 

Dates of visitations to the McKay are listed in Table ‘L. Other schools 
scheduled‘ are Harrington and Storrow. Additionally, information regard-— 

ing the resolution of the Peer Process and final: ‘feedback sessions will, aA 
be. provided to the visiting on-site evaluation team. upon “request, ‘ a 
Table 121ists the steps in the Peer prceeres See Table’ 13 for description ; 
of other evaluative procedures. + ae oe A 


: TABLE We Ks 


f MATERIALS MOST {fELPFUL ‘IN PREPARING INSTRUCTIONAL ” ee 
ek ees . ° STAFF MEMBERS FOR IGE ae oF : 
a , ante - z 
MATERIALS, . oT RespoNges 2g ft PERCENTAGE bye 
. 1/D/E/A., Handbgoks oa: 
- Filmstrip Sets ‘ a 
Films — : 
. e Nameé «+ ~ _,0 
. Implementation Guide 
- Other 
a IMS Materials 
rr 


The filmstrips and handbooks are avpilable to the building. staffs. They 
‘are, with a. few exceptions, kept in the individual IMC or: library.. When ° 
principals were intemviewed and asked, "How are they (insé@rvice media) 
used?" theér responses varied from We refer to then constantly!" to 
"They're never ‘used.". On the whole,:with the exception of one or two 
schools, the I/D/E/A inservice erials appear to be used very 
sparingly. While two school pMcipals reported, that, program growth 
See was being age Sati by rb to the outcomes (in the Implementation See 
‘  “®  Guide), the evaluation team did not fand’ evidence that the inservice .. 
materials were being used in.a planned program. of’: ‘self-improvement. ° 
The school staffs appear to“ See very little relationships between the 
concepts and practices illustrated and taught and in the media. PaChaabe 
/ * and ue problems they faced in, ae ere, Guided’ Education. ‘Bs 
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SCS RS RR eS 
PEER PROCESS-OF-EVALUATJON . 
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. 
» Outline of Evaluation Process 


ie Pre-visitation PSST TS tee “G ‘oo a 
" ApSelf-intentory © ue SP 
: ee bi Preparation of questions, . problems, - concerns or \ is 
% - .. -achievements on which local School wants feedback ee a * 
c. Conference with visitation: team to agree oh. te a: \. 
Me * . contract . sh re) Tt oe : 
4 ph Ne - BY “4 | 
“Visitation oF oy | 
Visitors: hold strategy session Sache tandem . iets te 
ae -A, Consolidation of data Bo Coane rs fae #. 
rd ~B. Assessment and evaluation, ~ ;_ + 
aeictae . ‘ er 
ee $5 MS 5 C. Preparation of strategy for ‘FeedBack’ conference ‘i oF 
oe ty Feedback Conference ; ba) Set : , 
; yale Contract Fulfillment a ee OS tie 
‘ i 
a ii 
‘ wy 
| 2. Draw yarger inp rations. 2 - roe 
“VI, AN League aig: of successful rae tiie: | es 
iar ba) are a) ; - eae 
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as Progressive : evdtae ono fro ect Lass que Independent. Audit 
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Drs, Vaughn .& DUP | fiana University 4 


The independent /¢ ¢ tae de signed to ‘provide feedback to 
the League faciTétatons 
zing, and implementit idividually Guided Education. ° 

. Implicit in this py essive evaluation is the premis® - 
that the program is ‘fn'the process of hecoming.. This » - 
typeof feedback process serves as a guide for future . 
actions, The team will visit. all schools, collect data. 
through records and administer questionnaires. The 
final report will ‘be oral and written. * 


] ’ 
i Principal's Peer Evaluation Process 

eague rincipals, MEC Sta 4 4 
Principals maintain a position x Jeadership in individu ] 
League buildings implementing IGE. They are directly. in- 
volved in an ongoing staff development process of train 
ing and renewal... Significant to the success of this p 
gram is that the principals are able to "take stock" 


esponsible for plarnhing, ee 


through a self-inventory and analysis .of the progress of — 


implementation of IGE in’ the individual buildings. ' Each 


building in -turn is vistted by. the team of thrée principals © 


who assess the strengths and weaknesses of the operations 


of.the building staff in its various components , (Instruc- .% 


‘tional Improvement Committee, Learner Program, Unit. 
- Planning,.etc.) and compare perceptions with the self- 
- Inventory done by ‘the building staff itself. ~ 


III. Title HI On- Site Evaluation 


the 


| ee 
By es . 


tle sitation. Team ' : 
A fivecnenbér evaluation team, designated by the Title ut 


in \ Supervisor of ‘the State Department: of Education, visited 


the Merrimack Education Center's Project Leggue 


ah, March 13"and 14, The team members will gather dita through 
of “i «q interviews with schools and MEC, staff, on-site evaluation, « 
“Sythe team members who have the opportunity to meet with 

a of 1 school superintendents, and community Yepresentatives - 
f ¢ as WRIT as principals, SSAEDETRS and students. 
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IV. Wisconsin Reading Design ~ ei ee Meer ce ee ine 
~-~ » University of Wisconsin a “6 ras de inch spac ctpongestiopsn! 2 OL 
ye ‘| oy Se sae es : 2 . a5 me he : a 
-Project League schools ‘utilizing the Wisconsin Design’. = |. "4, 
'. “for Reading Skill Development are asked.to fill out. : 
questionnaires regarding the first year's implementatfon , 
in April. The University collates the data in the . , 
oe ie «Type Il Field Test Schools. across the Nation, ~The: 
am i results, as.well as the questionnaires used, §n'the 


. 


- ~. gampling are available to, the Merrimack Edycation Center - 


a 
Oe eS CW ae nye 
. s 4 


af 4. for analysis.’ 
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' MEC for its ongoing evaluation. These questionnaires 
- include student ,fnterview data-as well\as self+‘ ° 
assessment teagher data, The IGE Monitoring Instruments, — : 
_ utilized.on a-sampling basis, are sent. to all IGE schools. , 

.  hationwide. , The Monitoring Instruments can also be _ ; a 
used by individual schools to measuré the extent of the a 
jmplementation.of IGE Outcomes, Subsequent administra- uh 4 

“tions can then be used to determine ongoing progress. : 
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“ “Record keeping ” Up-to- -date file of League’ events - © . 4 
‘ cand dissemination activities, i, ee 
” Peas, = & ~ *. been ha{ntai ned ea tae 
« orn .* . 2 
"On-site observations AN1 schools have been visited at Bee” 
Peo Fa e, oh « N .) least twice; Other visits'have- ; a 
a , | _ been made upon request. 4 - 8 
a On-site observations to Suecess ful practice forms. have . page a 
_~ identify new practices.., . been developed and data eet gic Rag 
. school within the y has been started. bh weil es 
school - - ‘ 


AY Needs agecesmant at aisinae tionnaire has- “tye a sai 
» » end' to saine population a November and wilt b mintstered 
«again in May. DataTs “Stored at 
. present in-computer cards. 

Indépendent evaluation A contract has been given “ee 
o Indiana University Center for 
“ : ' Administrative Studies and. an‘ 

g — interim evaluation has b 

vw”, ‘completed = ~ . 


“1/D/E/A il _ Each of the schools, ha partici- ea ae a 
pated by completing -t@ficher and .. 

iy - student questionnair 

available on the fal 

monitoring, ™ 


“~™~ be 
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ions, and participant-opservations., Teacher, principals,, and nd ? 
‘p#rsonnel were interviewed; questionnaires were completed by a’ |. Fg? 
‘ of teachers and unit leaders; principals were’ interviewed. forests rd ae 
4 a and’ Jnformally; MEC personnel were Wat olla and provided certain * 

ak office records; the evaluation team-visited/unit meetings, AIC.meetings,. « \- 

ny ‘nbs to interview staff; and, the evaluatjon, eam attended two EPOHNCE ‘- 
. League printipals’ meetings.’ a 7 


‘ 
* - 


— anf . for this. report. were. fahttad: sion ditcbortets Abwrtonadees: 


Each objective was *subcategortzéd, into elements that could s more. " ’ _ tig 

,easily measured than.the larger objective itself. After the deTineation : _ 

of important elements, the evaluation assessed the most desirable way oa 
” to gather information relative» to gach element. . The selected methods . -’ 

included examination of Merrimack Education Center, records; interview oo. oN 

of Merrimack Education Center personnel and principals,. questionnaire Bao 

- administration to al} teachers; and, observation.of a unit meeting and | 
an InstructionaY 1 provement Committee meet ing in each Senbols Phe SS 


The date for the evaluation was gathered as “follows: . hae ots é ; 


" Objective I. To provide Implementation Training Programs: foi , 
All League Schools (staff development) . + ’ 


1s Exentnation of Merrimack Education Center records 
. °2. Interview of. Merrimack Educatidn Center personnel . Roe tg, 
a 3. Interview of each principal ~ a ee ev oe 
4, peas aae of quest fonnaire to all teachers dpe va de ae 


Objective I], «To Establish a League of Schools : 7 7 %. | 
Eo (butTd an information: exchange) | Oa ¢ 3 


‘ . Examination of Merrimack Education Center Tacos: big! ts 
* a2. bial aad of Merrimack Education Center - Personnel page io 
‘ nterview of each principal ; 

x. > By, Administration of questionnaire to alt: teachers 


i : . . 
. ’ a Pe * 
} ‘ — ‘ . My ae ro . 
- : 
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Objective III, To Assist Teachers: dn Developing ‘Skits 
? _. Related to an IGE Learning Program == 
nner ieebedes etc.) 


1. Examination of Nerrimack Education Center record 
, .. .  @ Interview of Merrimack Education: Center personne 
(»* - 3. Interview of each principal -. 

. . . 4a) Administration of questionnaire. to all, “teacher's” 
ih ' §,.. Evaluation team Observation . 

ar 6." Attendance of Evaluation Team at one unit meeting 4 ae 

‘ . in each school . . ‘ 


: e > , 
‘| ‘ +e 
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| nsectve IV,: To Honitor’ Assess ‘ahd Evaluate 
we _ Implementat fon of IGE in: Project League 
We - Schools: : 


» te ‘Examination of Merrimack Education Center records - 


4 


‘ .. 26 Ipterview of Merrimack Education Center personel 
co. 4 93. “Interview. of each printipal . 
ee ee 4. Administration of questionnairé to all teachers 


oa Be Arteria of Evaluation Team at. one unit meeting ; 
yb) “in each school . 

. 7a, Attendance of Evaluation. Team at one 1 structional 
a eM eid Committee mere, in. each ‘choad \ 


2 Bee i 5 A ny 


4 
t 
; of |  Effectiven gs: xs aa ee of. .y he. 
} a In ividually Guided E ucation, ‘in order to be sudc essfully implemented 
! - + dn Schools, depends iff large measure upon the willingness of school 
:  adninistratorg to cpange their perceptions of their roles‘ in the area 
_ Of déeision makin Conventional school @dministrators typically 

| “°_" Anittate decision$ to which teachers or learners accommodate, and the 

: P- decision is then a ol bagged by administrators of.the school board. 


; The IGE concept mandates that decision making at the building'level be 
wn collaborative. That is, teachers and’learners must learn. to .use -arwide 


-variety of decisions germane to school operation. Rather, principals 


tend to help teachers and 1 pupils learn to make better decisions. . 
—— Peet ee ee ——_ San: ene seen Merge ee a ‘ . 2 
, The data suggest that principals: “and other: administrators ‘have racelved 
‘. information from MEC and the League. However, evidence that they have 


\,'« communicated effectively this information to classroom teachefs is \ 


absent. Without information, decisions cannot ba initiated. Thus, 4 


j we a ‘real sense, the oi process is being subverted by lack of information '~ 
e building level. . The alternative is perpetuation:of . 


» dissemination at t 

ee ~ a Ue ee Rieck building Nae by virtue. of the dependency ms 

‘.** - Created ‘by principals in terms of information-control, motivatton,/and 
ek * «thus interaction-influence Frce the top to the bottom of the pyramid 


y In effect, IGE eeevas juse of the broadest range of evaluative tec 

“s _ niques. Andy they have been used; ranging from the descriptive tof. 
_ , sophisticated, inferential techniques. Fortunately, a.number of fhe 

-. instruments and Strategies were already developed as a part of th 


% 4g? y . Of Wisconsin-and 1/D/E/A-Kettering on the IGE coficept. Data arg avail- 
- « . diversé data which wifl be available at the close of. Federal funding, ., 


are extended.. IGE, at this point in its development, is. meeting ithe . 
* criterion of a viable alternative to the wena e ional. : 


; = 90, . 
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ba _ +5. Evaluation of Team -Observation j bag Sat 


original research and development activities conducted by the University 
able’ to. support ° paige: on each of the objectives. . The accumulation of 


one year from now, should. provide concrete conelusions if present trends 
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bi 


- EVALUATION FINDINGS As Ng, Oh Mea” git fia 7 aes “a \ 
ie en “Individually chsh sahentowas an: approach: te. eases schooling... Et NE ee 
“ f..*  . that provides a framework or process.;for individualizing instruction. . 4 

|"... + It ts achieved through an inservice program which is designed to re- . a fo 
‘organize and redirect the time, . talents, and energy of all concerned , » *° 
with the educational process. . To this,"end, ‘the.Merrimack Education ‘= = -. |". 

Center undertook the Tohwation ofa League of Schools. Sngel Fie: = 

ahestives: included: * Aig! abit | : 


‘@ sitatrnent my ‘alt Fees schools of peer “support a a: Cas ‘ ’ [ 
0 establishment of a "communication network , ' re sy e, 
e provision, for iresearch assistancé and service, support 


‘In ‘adattion, implementatfon of ane Tce F Facilitator out mes sa an’S 
overriding goal. \ ea : f 


.. “ 1 hin i 7 a B, Pa PD een 
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oe Summary ae alan ae BP an Cigar aes 


"The assessment snrommation sal a need for exaitination by MEC ° Be 
2 eS of .the fol lowing fineters in their operation. ee * 


. 4 ; le Opérational’ problems and ssilite seem to be focused, upon 4 oe 


content. or substantive: subjects, Empha$is of IGE is based‘ upon pro-*,, ’ 
“vision of 4 productive way ip. hich units of es can function - \° ‘ 
together to meet set’ objectives. .Problems ide tified as Well as dis-° . © % 
cés@@ons. reported sugg@st thdt a low-level Know] edge and. Ton now exist ‘* 
_« | of Individually Guided Education as a process of interaction now exists: “ag 
“| after one year. Further, that impetus upon, given content areas has been aa 
and is presently frustrating by virtue of the inability. ofiteacheys to Pa 
. \employ the IGE. planning system and the Unit: og ie sates. in confronting * Be 
. \and solving operational problems, In short, IGE is perceived to be a 
~ "what" pather than a "how" and until this misconception can be clarified 
-and corrected, implementation of IGE will be effectively curtailed, 
eacher anxiety : and hostility increds 48Q0,, ‘and a.generally negative 
attitude fostered on the part of many roject League schools. -. “ 


a 4 a > 


aa \ There is a Tittle evidence. that the IGE: outcomes for principals, 
‘| “teachers ‘and Unit leaders and-Units--stated in terms of processes to be | 
‘ .  * achieved--have been used as effective tools in Sutetng yeRENENS toward 
sie full implementation of the program, 653 ; 
 ¥ t = ‘ 
aa 7 2. The IGE implementation. strategy appears to have been devuvtavad: ». @ 
. ;, to coincfde very closely with the hierarchical organization seen in ‘ 
- -'\" most school systems. That is fo\say, the,dissemination plan took into — » 
~ .' Careful account school board members , superintendents, bailding admin- 
E diaiae and to some pxtent, Unit Leaders, in-that order. ~ 
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While. there. $s. no quéstion that ‘school boards and superintendents eed 4 
to. be; informed, and that’ these persons’ have the responsibility of 


-\) ...-.monitoring. and:assessing..the effectiveness: of IGE programs being carrtéd. =e 


on:in their. districts,, there is some quéstion as to their cehtrality , 
' jn an eae ‘implementation effort. IGE is a learner-centered process. 
As sich, the ultimate sUccess or failure of the program is dependent . 
: upon the: attitudes, , abilities and behaviors of teachers and pupils, °° «© ~- 
4 A well-informed and_ ‘positive administration, while desirable, ited 
enNny out ‘the IGE. program if teachers are. ‘not informed and re iad 
4 . ‘ pee et *, 


conclusions. eal 7 < Ey 


te On the vaghs of the data ollected as part of this study, we 2 have ‘drawn +. 
q the. Follo ng conclusions wae oe 


4, Merrimack Eduentton canter i ce the: jisieeaitias of IGE 

“and ‘the league strategy on diagnosed needs,” Their.needs assessment _-—s— 

‘ survey that, indicatedia need for individualized instruction meas A 
‘ appears, to have been eee Beene cle tae ‘interpreted. ° s 


2. Merrimack Education Center has faforsei their region that. they 3° 
‘are an intermediate’ agency’ for Individually: Guided Education. This was. ~ 
accomplished “thrdugh h Cive-Ih Conferences, school METERS and dise “a 
Setbution of descriptive literature, bs a 
>. 34. Merrimack “Education Center has organized and. has Conducted che 
preliminary: inservice for the thirteen schools in their fiewly-forhed me 


" > 
Teague. ae Ws et ate “ait 


& 


4 
"Eos = * ® 


: 4. ‘The league has ‘beaun its initial attemgts. in sSoniitg support, , “i 
~ -a\communication ‘network, and a source of self-help. While only a few of = — 
the principal, unit leader, and teacher league members can actually state 
. “specifics. to document this, there is still evidence to suggest the v 
progess has begun. League méetings were’ numerous,- the topics were : 
“appropriate, and: there’ was an,,opportuni ty for input, feedback, and inter- | 
ep ACTION. However, the teachers were not sufficiently involved. It is 
‘. \Of dubious benefit, .for example, to hold a conference on the topic: , 
opr blems, Needs, Concerns, Resources" ard ¢irfvolve ‘only superintendents . 
’ and principals,,,, Superintendents’ and principals met to share successful 
practices. It might be ‘assumed, given the role of participants, .that 
‘the Successful practices shared related to admtnistratton. 
* ’ 
F ‘5. In,the eyes. of the Teague pembers, the ntntaunedinte agency" y 
and the "league" are synonymous terms. Because this year was the first 
year of ‘league operation, teachers have “had little id pate to visit. 
other schools, serve as consultants, and attend inservice sessions. 
Au sa influenced meth ‘perception an the lis mt 


. LY 


‘a goad star 


.. ‘the principals and 
s appraisal, it would have been interesting to gain an. accurate perception 


‘for the-principal or no changes in the sche 


(} 


x at Ree er that iis beaut fad’ nore. visibility: among 
principals, less among Unit Leaders, and-dramatically fess among teachers, |” 
Clearly, teachers, have very little understanding of, what the Leagud’ is-- 

_ and of what benefit it is to them." y : 


' ‘ 
2 


rough the eotablishntnt of a dhiagentabive: HUB ‘Committee; 
s"beén made toward the development of a strong ‘central 
J1- match the needs and resources of the League. While 
it-Leaders gave the HUB* Committee a-.very positive 


committee. that 


of the conmi tthe: ereoug the eyes ef teachers. | 


7. Project ‘League. demo trated extensive involvement of principals 
in League activities, Principals’ meetings, agenda ‘topics (e.9., "Role: 
of Principal in Individualizing Instruction," “IGE Monitoring by Princi- 


‘pal and MEC") were appropriate. —  » 


8. MEC's plan to train parents is to be ilbndeas The » deine els 


generally had a very positive view of the support potential of Parent 


Advisory Councils and MEC's.efforts to strengthen parent Support is 


commendable. 


1 


9. The: data suggests that the dissemination strategy used: by MEC 
in implementing IGE was hierarchically orientéd from the top down. The . 
Glue-In, Overview, and Principal-Unit Leader Conferences were held for. 
persons occupying administrative positions mainly, and did ‘not*involve 
those persons most critical to success jin the program-~ teachers-- 
until late in the sequence. This ,probably led. to feeffngs expressed: by 
some teachers, that they were being asked to implement a ica wines was 


not the'result of any initiative of theirs. » . 


10. Therinitial’ inservice workshops appear to*have been adequately 
conducted. This includes the Overview Conference,: the: Principal- Unit 


- Leader (PUL) Workshop, and the Preschool Workshop.. Teachers in-general- . ° 


expressed good feeding abous these activities. However, too much ofa 


_ burden was put upon principals and Unit Leaders in préparing the remainder 
of the staff for implementation of. IGE. It is unrealistic to believe 
that a short PUL could thoroughly acquaint a large group of principals 
- and Unit Leaders with the concepts of IGE; it follows than that it is 
, also unrealistic to believe that the principals and Unit Leaders could ; 


do an optimum job ofF. reper eg a full staff for: IGE. ¢ 


11. A general’ lack lof understanding ‘of the entire individuatty 74 
Guided Education concept was also evidenced by\some of the IGE princi-‘)" 
pals' responses to the question, "Should the preservice workshops be 
modified?" "How?" .Respondents who sat +) that one day of training 

ule would be desirable can’ 
only be perséns with unusual. training and knowledge--or--persons who 
were not acquainted with or appreciative of’the depth of the changes: 
nBiuaked by Individually Guided Education. 


ie 


tes IGE support materials are designed to be used over anu over, in many ae 


12, The majority of the league schools expressed some concern about 
the frequency of visits from MEC personnel. They similarly expressed | ra 
_some concern about. being able-to ask-for’ assistance and have the request - fo teh 
answered on a fairly immediate basis. MEC has ‘attempted to maintaina / /-° -: 
Tow profile INGTON Pay that ‘is, to ane the oe : fi Ma 
Agency Tole. bg, Aya 


Cl aed 


13. "Each IGE outcome - is discussed in-more than one media. The 


ways and,as a ready reference to achieve specific outcomes." 
(Impleméntation Guide, p. 18, R:10/71). -There is. a lack of ‘Sonanieney: 
between the statement above and MEC schools! staff view of the value of 


continuing to work with the inservice media package, More. than one-half «* ° . 


of the filmstrip-cassette sets, and many of the printed’ publ ications 


‘were not even available for viewing last summer.dnd schools ‘that did not: 2 Pes 


',s pursue their study, of the iain media have not. viewed the total oer eee 


<i 


elementary schoo}s., 


Sean aspe 
~teams, unit lead 


program. 


14. By ‘and large, MEC schai attenpted to initiate IGE 6: the con-. oo 
tent areas of reading and mathematics. In many respects, such decisions a 


_are seen as A alae iti ‘Reading and math are perhaps of all content 
n 


areas of the elementary curriculum the-most standardized. Broad goals, 
specific objectives, learning activities and assessment techniques--to ~ 
say nothing of norm-referenced tests--are. all Reneeay available in most: 


Thus, GE’ units are effectively able to chvcumivantonnctiiatty 


implementing t the IGE Planning System. ‘And, unless and until Units _ 7 


genuinely experience goal setting, resource teacher sélection and 
responsibility, role assignment, pre-assessment, grouping, scheduling ‘eg 
gn all of the daily operation concerned with unit instructional design, 
ndiviudally Guided Educdtion-will remain an idea or an: acronym al 2 
little or no meaning in terms of Venchitnige learnings, 


z Furthermore, the data imply. that pe and, math: wer 
building-wide and raise the issue of. what level he instructional de- 


 Cisions were made. at and by whom. . Building-wide instruction in’one or 


twa contént areas! is not advisable by virtue of potential competition \ 
for scarce resources and facility utilization, — , es 


a 


t of the program, The teafRers are organized into — \ 
rs-exist, and pupils have ‘been assigned to the units . 
on a multiage basis. © 


15. The sett of scaporiant appears to be the ihlost wideuen tuts 


“EXPORTABILITY) =) ZS 9 SEY, 
an ee aes _ Degeribe: al eutra quipnentand materia aa rete sp 


ear.) 


'» 


. 2 a. Introduction . Yee eon 4 
ane AW basic. eléments and innovations of ie project are 
. Se a “considered totally. adaptable and axpoy able. Titles 
0% - "are included in Table a . 

q * s peat te ene che ; ’ 


b. Materials ‘aia Equipment. d 


Mi “~~. The 1/D7E/A-Kettering materials are’ ‘available for a 
1 fas, e', > >) “service use in-each of the fourteen schools. ‘In 


aoe me 9 _raddition; instructjonal programs that meet the criteria ~ 


~ oo. .. JS of the IGE prograif model are available for purchase. 
r aan ." “These ‘include: Wisconsin Reading Program, *IMS, and 
Continuous Progress. Learning Kits. Each school has 
:" .* ftom, the lech it the normal allocation of resources @. 
‘ a \ “from t chool district and selection are made from” 
ws... thesé resources for sig and. pear ica 


oo Nee 


2. paser'ibe ay key variables without” wich the progran.shoud not : ooo 


ae . oad ; be attempted, 


F 5 er ¥ ‘a Context of secant Bs 


ae °° Merrimack Valley in Northeastern Massachusetts. 
' >; Ta ~ fourteen schools’ involved(include -a variety of. 
built from is 
buildings in 
’ levels K-6 and range in size from 300 to 900 pupils. . 
; One of the schools is 1-3 only while a; “second serves 
\ . j only 4-6. »- ' 


es... * 2 
lude “grade 


architectural arrangemen 
‘recent as 1969. Most of t 


et _: ‘be Special, Factor for considerattdn of. Adoption 
] : @ Proven ‘plan for indivjdualization of instruction 


e(Curriculum programs are. avaifable commercially 
, so that.an instructional program designed to meet 
: » . . "+ the needs of the individual sean ‘he implemented : 


@ The organizational plan for * he school is expticit 


wane pattern and the school-wide decision-making 
y ~ eo * 
® The program has . ‘developed a comprehensive {nsarvics 
; 3 program that includes outcomes and accompanying 
a multi-media materials ° 


*IMS = Tastructional mi Systems 


~ v 


fr a ns s A : : . 95 - 


bee 


“age z * Project League serves ten communi ties located in ye r 
The 


and roles-are clearly defined. for the differentiated 


id 
e ‘ 


“e The cost of fighonuinns 4g  velatdvely’ Tow. in 3 
"and within the reach Of. most.. schools © Ry S 


# igi '@ The innovation is. essentially the implementation . a ce 
of amodel that facilitates. ere nae, Bh ye 


° 
ror ome Se ete tered 40 


f. instruction. ~ Because it doesfhot include particu- eG oe 
“ae _.'~ Jar instructional programs orSpredetermined group- = 
‘sgn Pe ing ‘sets no decisions are made acegrding to local, a ae. 
» q _h, weeds, he es eo i 
OG Sr: , a Ae Es a 


» 
é » 
, P ge 8 : i 


7 a 
- * rs . 
3. yA "ite “S boa et Cg ale FOS ee Pe 
ee a ‘ : ; ,. ee be, BRE ae A eae 
er & ~ i oy 


fa 
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* Describe. the appropriate paputatfon for this project | 


/ 


a “Population boy i: ain ye F « — a 
The target area’ includes fourtgen schools in en the 68 2 
communities located in Northeagte#n Massachusetts.*. .- ® .-. ® 
Most of the area is suburban with one urban school + GE ate eg 


The student population is 97% Caucasian with 3% ac come’ a 

Black and Spanish speaking.: Special. educatitn eae 2 mn 1 ey 
students are involved.directly .in- the program. a 

., 7 > 4 Childrenmtypically in Be ao ‘levels K+6 areaarietis ee 
ride by the project. _ » iia ee 


ia "Be. ‘teenie ‘fe nature. of. ‘the Recitation ere (@.ge5 at ety! 

the school adninistration,. teaching staff, physical facilities) which Na ae 

_ you fgel would be ids to' the success of the project, ih an re Sie 

district. | | ‘ ee 

iets ’ Ge oe ais” Se  ™ a: eee ak ee I m4 

ety Be Activities yes ee eee re. o : or 
The ‘peoreanteation of. the school into a * multtundt Tie 

school with. differentiated staff with: hole description gh OF ae 

and outcome expectation «~ 1 a ea 


° 


P Inservice program at the "Teague" and local school - a oF | a 
ate Ss yn em to provide the necessary skills to.d velop cand | et.” 
= » manage ‘an ing ividual ized instructional program: . The ~ ae 


a , _ Development. of a resource exchange and other ‘appro- se ee 
i a eer communication channels at various. 1 di ae og a pee ey 
ne league schools te Bt ds, (Aielgh <3 


i ag ee ee arg en 
be Facilities . =" ah aa ee Deb) te. a? Sey PE ae 


Bas aa es 
Ten éommun tied, raging in. size from. 300 to. 600 ‘ak ee 
1 °° students.” ‘In some of: the \buildings ther i$ Mttle © |.’ aw, 
remodeling, while in others, floors are garpeted» = ss * if 
ane flexible Spates ma abe aval lables oa i. er San, ae 


: “TABLE S14 
i) mo a 


“ee MATERIALS. 


,% ree § 4 

f 
Y/D/E/A has pedunee a savtaty of materials that wilt assigt, 
school staffs with implementation of IGE, Each. of the agar 


outcomes of. AGE is discussed in more than one media. owen” cs a ae i 
é PD ree 


One} At. A:T a teethey . a eR 
% ‘Tuesday : hd h sa 3 : bby ere 

* ManyRoads, ae a RR es ae 
‘+ "The: Unig Netting. 4 8 Re 8 eh tt 


Ry io nb : . ; , : y . Pk 
y - Filmstrips ‘eM ae Die (te * ORR 8 te a ' Fs 
The IGE Learning Program | goers wh ed 
Organized. for Learning - ft ea : 
 FRIGE Learning: Modes =.“ ' : i ey 
« Performance Testing and insarvation ty i. © 
~The IGE Planning System . ‘eee 
‘Building the IGE Learning’ Program a ae 
A. Reach for Tomorrow** ~ : 
League Linkages**. ; eS 
; ‘Home-School Communications** wot Re ada 
.« “IGE Implementation** a i 
‘A Managing the. IGE Learning Environment** Ve 4 


\ Print Dotuments A : : 
IGE Overview Brochure \ TD. 4 REE Cae al 
|. _ IGE Unit Operations and Roles Sas 3 “pi 
IGE Implementation Guide _ M, = 
IGE Learning Program MB. ae: Pie si 
IGE Principal!s. Handbook ,. ay ange) EY Ne sore 
¢, IGE Assessment Handbook : “3 
|e. > Color Me IGE °- ‘ eae CLE OE 8a: 
-'* +s Multiage Groupihg** ~° © oe * 
t ' Learning Styles** ~ W< i osigh. 
- IGE League Handbook**. , a sie 


aS NN Fe Se. a 


“y 


i New ee available in 1972-733 not.all materials. 
were available in 197-7220 5g ee i 


* 


| Wisconsin ‘heabaeeh and Development center for Cognitive Learning 


QO 


Totai Individual ly. Guided Motivation Package A A : 


Andividual? Guided Motivation:’ University of Wisconsin ™ 
te? publication . 


‘Guidelines for Implementation:: Guidebook for. planning 16%... [. 
_ Three vs. on my Tilustra 3 + how IGM is impfemented in a Unit 
: p * wasconst Pee Hes Reading Filmstrips .. 


/) Wisconsin Design Meeting Leader's. Guide:. Inservice agendas. 
_ Design for Reading: Introduction. to the Design | 
~- Tnstructional. Programming Model in IGE: Delineates steps ‘in. 
carey ng put t e,program 0 TGE an iMustrates their 
ides et ae ~s 
skint s_and Objectives: Emphasizes the nature of benevicral® 
- ~~ objectives written for every skilt in Word Attack,, COOPER 
chension, and Study a ane , "\. * 
Assessment; | Find Need: ‘Shows two types ie assessmerit) 
ates ne that are part of the Desi } ; 
The’ Complete Picture: Appraising Individual Rerformance:: - 
Considers types of madiyidual testing and observation as. 
they’ are used wtth the Design 
A Matter of Resources: Shows the purpose and organization of 
the Teacher's . Resource File 
Preparing for. Insthuction: .. Demonstrates: the "peepee tn of 
he pupil profile cards and illustrates. the use of the. 
. cards for formtng, instructional groups : wh 
Dasign “in: the para ental Reading Program: Presents the 
| “aspects Of the.overall developmental reading program ' 
(developmental; torrective, remedial, and accelerated) , 
Focusing Instruction: “Stresses formation of subgroups. within. 
each’ shy Troup to seconnaast individual needs 
Stud ski Ths: ning to Learn: Deals with the aspects of . 
arent that differ from Word Attack 
Self-Directed, Interpretive, and 
: how they may be integrated 
, @ reading program 
prof @ Cards: A New Card Game for: Readin : Presents the 
i design to students in terms of A aegis they will encounter; | 
testing to find out what they a meee know and what they _.. 
“need*to learn,’flexible g¥oupin help them learn. one. | 


~ thing ata time, post- hosting for mastery, a setting goals 
; _ for progress: i oe ae : 


ee, 
ie ' , 
beg Yo ‘ ‘ hee 4 
* . * i \ 
% : 
5% ra A. , \ 
. ’ 
* ai ' . bs . 
. “ ’ Nene r BP oy ‘ - 98. # 
. ee 1 
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ee 
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udiov. sua} Package: 


hm "IGE for all Children" ie 
Et ny “Filmstrips accompanied by audfotape 
bi cassette -= *IGE/MUS: Rules and Responsibilities | 


* 


| 


" 1GE/MS- gM 


| 


. 
4 


iTable 14 (corittnued) 


>t *Instructional Programming in IGE ‘ cai 
as Saar One set of ten: transparencies and a guide ; 
? escribing the instructional programming madele 

Ise Print Materials Package: 


if ke | ee *Five IGE/MUS Glidelines for fantamsittalisn , 


“*Five IGE Simulation Materials: juchtines’ and ° 


~*IGE Brochure < 


of 
{ 
Color anton picture film 
*IGE/MUS. Organizations and Operations 


*Grouping Patterns F | 
*Assessment in IGE ; Hoo 


} a | 
t i! 
‘\s 


(Klausmeier, Quilling, and Sorenson) ‘ Ni 
_*One Multiunit School Directory . a | 
‘*Five (three to five day) prototypic Agendas 
*One Resource File 


yk, 


Grouping ie . , 


| 


Project League is amenable to aviatiAge. The godt of pro-. 


Viding individually guided education is a laudable one, and the Project 


under review provides a viable method of introducing it into systems 

which have such a-need.. Emphasis on staff development and the utiliza- . 
tion of existing facilities and personnel add to the ease with which the + 
major program components can be replicated. The availability of’ needs - 
assessment, and evaluation instruments contribute greatly to exportability 

as do the reasonable costs related to inservice materials, External and 
internal evaluation reports provide insight into the possible restraints - 
which could be antberpa ted ‘in eat and Mave hoping a similar" 
program. my F " 


6.‘ Describe the nature of the fnceatinnss variables (e.g., the ° 


* school administration, teaching staff, physical facilities) which you Z 


feel would be ‘critical to the success cal the project in an adopting 
district. _— ‘ 


e faphasts on staff development’ and the utilization of existing 
facilities. and personnel add to the ease with which“the major program 
components can be replicated. The availability of. needs assessment and 
evaluation instruments contribute greatly to exportability. as do. the- 
‘reasonable costs related to inservice materialss External and internal 
evaluation reports provide insight into the possible! restraints which 
could be piesa in introducing and: developing a som Ter program. 


“Fs Indicate "the willingness of the school district(s). to continue 


-the project and act as a demonstration site, This'is in addition. to oe 


"PEG eePleRPten by the-Schoo] Superintendent,” 


The lessening snveTwerant of Title III and the increased ° 
involvement of LEAs are indicative of the fact that the LEAs see this 
as an economically feasible project. Letters from all superintendents 
document that the LEAs intend to continue this Project into its fourth ~ 
ven | with local funding. her 

" The Project League” Somuaes Report contains the letters 
Ae all superintendents’ of all LEAs Spacek plecging continued moral 
and Financial support., 


The following sutdence can al cited: 


© In the first year, MEC (Title 111 raat | paid $20. 00 
‘ per day for participants in preservice workshops. 
In the second. year MEC paid $14.00 and the Vocal 
districts $6.00 


-@ When HUB members and on leaders attand néetings 7 
nes districts: pay for their substitutes Ms 


« $ ly 


Po 
| 


ar MEC. pafd $3.00 per pupit fee materials;. 
on year MEC paid $2. a0 and locals 


The school ‘commit 


" A 
_ we ey 
——_t___ ne Se 
a. line item a school budget and amount: ae | 
|gervices -- $25,000' =. 2311042 . . 4 4. ™. 
be direct in-ki d-service (please detait) > 7 Met Sb 2 
fer unit leaders, — : ee a ea 
sends. for ‘preschool workshop : ‘< ee 
‘ ay so teachers can attend léague functions ips gs 
iat materials: comparable te Gh. ee cycles | a » oe 
: ; £ 4 re r 
> alocer. REQUIRES, , Ne > ee 
In ‘completing this segtion, please consider the following: ‘given the oe 
fact that this project) ‘and related materials\have completed the develop- a" 
ment phase, what’ is the lowest expenditureof, money in each category oo 
listed below you egtfinate would he ‘aug for the start-up of we St Se 
project at another s{te? | ; ae es 14h es i 
OMe he i 
ar ae suet $103, 058. a a ee a eg, ¢ 
R ; N : 
trop First operational year refer ‘to Table 19 : 
, t ” { 
; F vy 
2 ig aw Pivh 
7 ) ; * ee: ee. " ' , 
bi ‘ * ay ates 
‘The Merrimack Education Center (MEC) provided, $10,9 0° 4 7 * 
“dn planning funds for Project League. There was no 01 Nn "  & x? 
ning grant from Title III. Almost half of these funds 4 


($4,500) were used to support the professional staff © yo 


during the planning perigd. Other funds were allocated . 
to sefd the Project Director to “Dayton to a Kettering 


Foundation Workshop’ and also ta bring in outside con- : 
ints ‘to work with principals and unit leaders in . é 
| Materials, use of facilities, and secretarial : =, 
ass stance was i provided by MEC during this Banu ; 


djupon a tehgue 


\ . Hy 
Reaiagispat of 10 parficipating donmunittes \ 


10s 52 foo thy 


nd ” 
3 ‘TABLE 15 
\ GL \\ gs CONDITION OF ACCOUN Ha 
ton, + TITLE TI «4 i 
~ EXPENDI-. Pa | 
TURE — CASH ’ 
% ; , + are 2 R 4 ye ; Fo & 
Professional: Salaries 5 a $32,400 bite te y se 
Nonprofessional Salaries 104 "7,00 ‘ " 4 . t 
“Materials & Supplies 106-1 =; 2,00 . * 
Books & Periodicals - 106-2 boo | Pe Ao 
Travel | 107 3,500. .° | : or 
ae Dissemination oo. io? 2. 7, aL bt - aot | ‘ ° . 
TOTAL ADMINISTRATION. $47 500° | a a 
tr mE samemememeenemed ‘ J ( 
Professional Instruction 203 9,800, “ i 
Nonpro fessional Li cP ner om, Din ae 
Instruction sa 204 ; 11,572 - Roar up 
Contracted Services «+ . Peed a @ se 8! See : 
(Consultants) 205 4,700 aa es 
Materials & Supplies. _ . Pa he : Oe oe Lo 
. (Schools) » 1 206, «18,486 oe " 
TOTAL INSTRUCTION My 6 $440 8 een ‘an aes 
Custodial; © | er wake 
Telephone 699 3,000 by Mh) 
TOTAL OPERATION OF PLANT $ 3,900 THK ie 
Buildingtosts =”, 705 spo . 7 
Equipment 4 RS 1,420 0 8 
RECAPITULATION TITLE, 111 $103,058 
a r w | ‘ 
en . big: IN . ; 
7 Sa EES 
ek oat noes rar 
a ' i ' ‘ y % z 
i ie ee 
y 6 “ ; - me - i) 4 : \ 
‘ eee | t > 
w “§ ' ¢ 
. ; i 42) . re " 
A ot Aer & Poe Bt ee AS fa aa oe 


if _ K . i 
: ae Costs* = . 4 ‘$. - “ 
eg  StartSup ait (one tind éosts that would be required to. 
: “ae “begin this ls as School ne 
(a, Salaries fF beey PE ak Re 
/ ‘. professional. __ $4,500 
: Nonprofessional : iS "¢ ~ 4,500 
b. Contracted Services te ‘ - 1,000 
_ €, Materials.& Supplies. ay 00 
a d. Travel (indicate’ | purp els) bg 300. 
as “Rs ‘Equipment ¥ 4 i * ES a 
f. Other expenses “(staff deveioneakt fe S50. tly 
/ oc 6Pental, ‘yoom modification, etc.) “7 7,000 8) ae 
: ge Total start-up, dosts ye os 4: 12,250 aay 
“he. Tota? number off Jearners involved ete. 500 
ta mobrovingte fbr of hours cee ee ae 
Initiation Costs ; eh MiGs age ee Te ae 


The first major thrust of Project League was ‘to ‘Join ten 


communities together with the goal: of inter-school’ distriet collaboration - 


‘for the introduction and implementation of individually guided education, 
“Much of the resources necessary to initiate the program are already 


albocated to the school and it was to a large degree a& matter of util1- 


zing existing staff and materials according. ‘ta: the. ia program, _ 
\ 


an oem 


Operation. Cost After Inada 1atfon > 
a,’ gf ahh ys 
‘ Some additional money ($23 per’ mutt is needed to. sspehaee 
materials that facilitate the individualization of instruction and 
‘supplement: existing wp at If paraprofessional assistance is not 
available then money heéds to. be allocated. A major rete of the cost. 


of the BVO eeE ts Bate TSvAby the local districts. 


Conmments va a. - es 

This cost is based upon: 4 schools forming a league and total 
costs would approximate $49,000 (12,250 x 4). It also mo ¥ 
investment by the local school except for in-kind. 


7a. ° 
a4 
rm “ 


e \ ' ; ’ 4 i . AS 
ee . | 2. Continuation Costs (beyond first year, if different) 
Rw § 4 a ' 
\ +l a, Satarjes sey 5 . .) 
4) + *. professional |. «tae ey 29500 \ aes 
Fa, ' Nonprofessional = \ \ 4,000: rf i by, 
b. Contracted Services ; 500 he 
, (+. (Give ‘general description) . . 
be as . Materiats. & Supplies 500 ee ? 


d. Travel (Indicate Purpose(s)) = * 


cr * 


Other expenses (staff ee ae 
‘rental, room modification, etc.) 


1,000... 
‘4,000 | 


Ew its ——— Equipment Bs " - “ t ae 


- g. Total continuation costs _ 


Per pupil. costs of have school — 
district: (9,800 —.500 | 


» * Te Ber pupil. costs i this project (approx. ) 


9,800 . 


J —EE ESE EEE ae EEE San ll 
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a 


~ 700 rd i 
20 a a 3 ie 


: hee ‘there’ other setts you night suggest of: more cost effecti ye 
ways ‘of installing. ‘this “project into other’ sctes?: : 


kieran quantity) * 
fi 


\ xe 


Comments 


4 ’ Above’ figures based ish initial fur’ schoat eng « senandad to . ; 
“tglude eight.schools. When and if aides can be picked up by the 1 cal ote @ 2 
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an a8 INDIVIDUALLY GUIDED EQUCATION 


Education). Merrimack Edycation Center. Project LEAGUE is. an 
intervention strategy ‘where change agents,/i.e., the staff of the 
Merrimack Education Center, ara attempting’ td: bring about innova- 
- tive behavior; leading. to individually guided ‘education (IGE) in 
a set of thigtedn; etenegtary schools. 1972-73. (MEC 000 189) 


opt ication for initial Funding: Project LEAGUE: (Learner Guided 


Education), Merrimack Education Center. This project will afford : 
a solution to the problem of developing widespread educational- 
innovation-in the form of individualized instruction thrdughout 
. a large geographical region. + The stated regignal needs include 
increasing pupil motivation, initiation of new instructional : 
materials, diagnosing pupil needs ,' and utilization of. multimedia — 
nS instruction techniques, ; ° 1971-72. * (MEC 000 :190): oS id 
Evaluation’ of revediwek ‘Education Center's Project LEAGUE: 1972. 
_ Indiana Univers ity/Center, for Administrative Studies.” Evalwation 
of a Title Iff Project on Individually, Guided EducsEI00 in ‘the * 
mul tiunit school, (MEC 000 007) 


f  evatuavion of Merrimack Education Center's Project LEAGUE, 1973. 


Shattuek Street School: 1968-75 nentavaiens it Scores. ittieton 


Indiana Mier SIM Genter for Administrative nano (MEC 000 070) 


AGE Resource People, Merrimack Education Center. A listing of NS 


Individually Guided» Euucation resource. people with specifio he gee 


expertise who may, be available for consultation dn IGE “issties 08) ; 
) **  (MEC @00 088) og Mee Oyeg Lae 


ro ai AO aw Sn 


Center, Study measuring effects of Individually Guided Education 
“in the’ Multiunit School on the results or outputs inj terms | ats 
openness of the schaol program, (MEC 000. 049) ne 


Project’ LEAGUE: Learner Guided Education, Merrimack, Lucsbtie enter. 
Project. League is made up of fourteen schools. in ten communities 
in Massachusetts; project concepts, objectives, and activities’ 
are described along with evaluation. procedures. (87p) (NEC 000 067) 


| Project LEAGUE: Learner Guided Education. Merrimack ‘Education. Cantar: 
A booklet: (9p) designed to.answer readers’ questions about 
Project League. Project League is a Title III ESEA project 
“Fee tn to oe IGE in a network of ere 
“(Mec 000 198). & . 


a 


Public Schools. .Test-results for alt thildren in the. Shattuck \s 
Street School from 1968-75 are given. The scores ‘demonstrate Aaa 
* in terms of academic achievement as measured by standardized \ 
achievement. tests. the IGE program is an SverAnE TING PUSAN Nt 
’ _(MEC 000 ee ; 
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. Appticaston, for Continuation Grant: Project LEAGUE shanner Golde 


o> 
| Introduction of the 1GE/MUS Model ‘tn Selected Elementary “Schools ae et 
the Effects of Organizational Output. L. Bernal/Merrimack Educa't ion 
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sane of leat : 


* Schoot Districts. 3 


: hereby sees “that the sian weinek. which 
is under my administration, will, if validated,. and . 

‘ 4f State’ or other’ funds: are available, serve as a 

State demonstration: site for a period of at*least | — 

: one calendar year from the date of notification and | : 4 
subsequent, receipt of: funds of such’ selection. €: i ae 


Ut \ er, oe 
i w % Bs ive ‘a ; 1 \ 
i “Superintendent of Schools — 
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A SLATED wm en! y ) % 


sa sik ee gave a fin cian £ a. snince tebe: 
th» upon that validation report we now offer a simulated version which 


states four objectives, presented “in Table 14 and shows how data is g 


collected and treated to produce a set of findings and conclusions. 
. , Since success and failure in the sgcial sciences are judgmental 


decisions, ‘the scope and‘detail illustrated under the "validation ey 


ropa i ia subSeCE TONS wrth follow could serve as. useful’ examples. 


When ‘val idationa-and ‘evaTuat’ on--are ohne dave dno the onset of an 
sist To tanovative project, the.expectations should be that the preaece will be 

better designed and that the.Jevel of record keeping will be improved’: -. 
; and will serve the prpayes sof accountability, Each project must. : 


*. demohstrate success or its tack to help assure the. Adoption | of only 
effective TnnGNeEtNe ‘programs, A 
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“ MASSACHUSETTS VALIOATION TEA neon 
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Prd. ect League: Learner Guided Education 
RA OF PROJECT ae on 


. KE 


Merrimack Education Center Ned 


wt 


on - Jsf Walter &. Krupa 


Dae isi Walter E. cut 
‘ Validation Team Chairperson =‘ 


? a A 1 ;  * t) F 5 a: 
ah : eg ’ . se" ks mo 
| Validation Team so % _ Address a _ TeTephone 
‘ee. IDPs Reter Maynard hal se © oh oo aes 
3. Dr. Gerald Work a! ae Plater oS E by 
4. Dr. Janet Cows 111 ie ae ee oe - \ E 


= R ae 


\ t's : ad ef 


A. ‘This program is certified as being validated as an ele mE, 
and'is recommended for state. diffusion 


airman, New England Regional 
Mae Or: Teeht 5 


ys 


B. This-program is certified as. being validated we la a 
® dissemination. 
arch 6, 1973. 
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C, This program is not. certified as being validated, 
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i i 4 ere ent nent nennn nr ann anne ante ter 
FELON aE PRE Tear es “ 
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éighoeae!® If a protege atta ips category A certification it aitomatieat iy” 


attains category CACTI ESET 


Pone tus ton and Recommendations 


The; team recommends that the. Project be validated and that every effort 


, be'made at the local, state, regional, and national levels to disseminate 


and diffuse the elements of this Rees into other schools and school 
districts. ; . 


« 


| lacs) 6, 1973 | c 
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_ | Reference where 
evidence for column . 
4 may be found : 


Evaluation pro- 
cedures including 


Findings and 
“<instrumentation.., | 


conclusions 


Data.analysis 
and. treatment,,;,, 


Objective 1a: To identify & 
develop leadership capabili- | 
1 “ties and peor 


Interview Project 
teachers, unit leaders, and 


isual Interpre- ||: 
i | . | Director 


Jv 
tation ° 


- ‘| administrators ma 
reetelae 1b:, To%select a a rte ar es eer  s ; 
and/or develop training Interview schedules | ‘Frequency ‘), \Identified areas | 1972 Evaluation. 
‘+ programs for selected.target questionnaire ee | of importance * Report < oe 
populations based upon need a * sgt ee a 


Objective 2a: To incréase. - Record keeping . | Visual) Intérs ‘Improving 1972 Evaluation Re- 
parent participatiorf and Interview schedule pretation. eae 8 port, Interview ele} ' 
support for IGE objectives : - Ss / mentary” principals . 


an | 


Project’ League ‘.. 
records 


‘| Visual Inter-| 
pretation ° 


Record keeping 
questionnaire 
measure 


Successful} 


ae Objective 2b: To maintain 
a “update an.educational 
«clearinghouse 5 


(r 


Objective 3a: To identify Outcome Item. analysi Uneven imple- \, | Project League 
; needed support systems and questionnaires ‘mean score . mentations | records 
~ yf services in implementation he ‘indicates areas 
of. the 35 outcomes of emphasis 


Visual Inter- 
| pretation. 


Objective 3b: To establish 
effective linkages with 

“|. external programs and 
resources 


Project ,League 


Observation ; 
- | records 


records 


« 


Objective 4: To monitor, Output analysis 


't test * Project League | =e 
access and evaluate impTte- Outcome monitoring f test records ’ 
mentation of IGE in Project Pupil Interview ‘ \ 


“mean scores | |" 
League schools . é . i 


yh pag th dy 


parental /commun ty involvenent, eerie) apie a 
training. ad h 


Validation of Evidence? re, ae ‘¢ i is tee. 


i. 
” A review was conducted of roject League! s, innovative Keigionenies. name 
organizational pontinecton, decision-making podelts. staff development, :" 
‘and count nued ‘coordinati de : 


t & : 


Although many s¢hools in Susdechusetts may be jnvolVved.in_one or more~ " 


the above innovative components, none oe Bh of Project League | 
are involved in a total management approach through. IGE. - tee ast ’ 
League effectively Dae La aes: Pempenenes ina re stems 
approach. "° i 
\ ; + 
The total number of IGE schoo}s in tho. Commonwealth of hassachusetts 
represents 12 of: ‘the tate's alone tary schools. re 


ve ‘Evidence: Nation; IGE bieeekors lists 13 Massachusetts schools, one 
adde opten ber, " 


1972:-= Total: 14. \ 


Massachusetts State Panter of Education, Research... : 
Planning, and Evaluation ESTs. lists total WaSsAChiserts elementary 


is ial at 1 2318. “ : 4b pr 
ee | 
: qs \ 
Rating of Evidence . . 
__ acceptable (k point) ‘ 
___ not acceptable (0 points) : Explain 
‘ Also record inappropriate box. i et 
iy > : j \ ‘ i | 
‘é - a ee 
2 f ; ss te : m / 
{ : ~.. . i 
a a ~nvise 


2 39 
ee . " ; 3 “hi qt . ‘ e & A : ; Fi 1 7 
2 4s ae if F j ; iy 8 
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pT oe fe ‘ , os fs 
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7 i : poe & 1 on “a 
E } ! ae iat " ip 
Mr, 5 ‘ 
“'. 4 Teachers and Unit eaders were aie to identity sha sesuiete that | ‘ 
, took place prior tg the implementation of IGE in September that were ~« + ee 
mgt helpful. oi shows the i a to that POANGR = ; 
; a : ah 
ved TABLE 18” ns ee a 
ACTIVITIES IDENTIFIED BY STAFF en 


AS MOST HELPFUL, .IN PREPARING THEM FOR IGE ” : ae 


* 


y : ra ant 
i. ah F : foe a, 8 by 
, \ 7 y, : tag 6 
i) De 7 a ar) q . 


~ Frequency’ 
of Response . 


Activity ,. . Percentage 


: ‘4: “nt 
a Ae ra 
aot 
a 
hy +. Vee: IGE Workshops | ; mt 
2. Independent Study : 
“3 3. College, Courses et 
4, None 
ee 5. IMS Workshops 
Other ee 
3 Previous Teaching Expetiterice 
ih; 


student Teaching 


Atteddance at. one of the HUB neetings denonstrated the urtactivenass of : 
the staff development component ‘of the program. .The schedules and 
‘>. proceedings,of the ongoing monthly HUB- meetings, “principals! meetings, 
'. and unit. leaders' meetings which seek to enhance the already identified 
* Yeadership potential afd which offer comprehensive inservice traifing 
. ; in staff. development are well nOCUNAR LAT. u 


| 


y 
&. 


To what degree was the objective Ri tas eas What. were the . 


dis skills learned? To what extent did they conform a ‘the 
ee, expectations epee tree.) in the bbjective? 


ee 
eee 


" Validation of Evidence: 7 4 3 
The Project, in its’ second year of operation, cure éxtensive . 

data to support objectives. ‘In effect, baseline data is. not yet ade- 

«2 quate to provide for definitive criteria in all of the. SEenINeS ‘ 

" Eyaluation procedures are axtengive and intimately. connected to project, 

objectives. . * ; *, 


F ; o~ 
Evidence: Sets of date for each of the objectives. 
Many of the instruments used were taken from pitepians developed 


| specifically for IGE by the University of Wisconsin Research and 
‘ "s Development Center for Cognitive S udies and Kettering I/D/E/A - valid, 


ge "Evidence: Project sauces bs tion Grant, October 1972, * 
av ” Septenbe er 1973, v 
> _ Systems apprpach being carried through to evaluation. strate- 
"gies. .Since Project is.in Si d years many of the strategies are 
+ "4 2% in the process of being refined’ 


a Evidence; \ Project League, “shiloation report from the Center for 
pe, Admin strative Studies, rs a ion of the IGE/MUS Model in 
“Selected ISHN e EE Schools." {, ‘ 


\ 
: Rating of Evidence Pe.’ 
. any e, point) > } ; \ o/ r 
a not acceptable (0 points): Explain : 
& 


. ¥ . 
‘ Also: record in appropriate box 


~ 


attainment, of, the objective(s).: “a 


.ciated learner change implicit inthe attainment of: the 


> ties. Examine the conclustons drawn from the evalua-: | 


Reviel the evidence that support 


he project’ 
that the project activities were 


cause of. the 


claim 


The evaluation supports the conclusion that the asso- 
objective was directly associjated'with prgject activi- 


tion evidence for the objective for the:purposes of . 
eee that the evidence super the probability 
that. learner change was associated with project acti- - 
vitieg; examine the conclusions to determine the 
(general izabitity of the findings.. 


Objectives failing the criterion mse be altutnated 


from further consideration. | Pty 
: \ af. 
bore “, \ gee at 
Validation of Evidence: : | wo 2 
At this: notre ti: the Project data ea been extensively analyzed.. The. 
‘most, extensively used were descriptive techniques--most appropriate at’ 
‘ this point in the development of’ the Project. In those cases where P 
inferential techniques weré used, the strategy: was appropriate, 
Evidence: Examinatién OF data connected with sh of the objectives. 
Sree 
. No glaring errors noted, , Data analyzed by. computer assuined 
to be error-free. \ 
Evidence: Examination arate connected with such of the bjectives. : 
At this point in the ect, conclusion must be considered 
tentative. ®ata available to sup prf\ tentative conclusions advanced. 
“idence: Project League, evaluation report from the C nter’ for 
nistrative Studies, "Ingroduction of the IGE/MUS Model. in Selected . 
lepentaty meneetss" ef : . 
\ ( : gee * _ yes a 
Rat ng of Evidence, ey a Be ot , ; . 
me (1 point) . 4 Mig i 49 %, 
not acceptable (0 points): Explain vit 4 
Also Senate appropriate box s : 
i eg 
: ; : 
¢ Ri ‘ Let wa 
‘ . -114- é 
E - / ‘ ¥ a ‘ 
3 ra : 125 


<TABLE 19 _s 


* ag AS LEARNING PROGRAM DIFFERENGES IN 
ie y ‘9n-72 COMPARED-T0 1970-71 AS PERCEIVED BY cinnotne- , 7 | 
ne é " . ~ “PRINCIPALS ‘ eS Daten 


OTFFERENCES 


We started the year with walls closed.’ Now a few are 
opening." Wemoved from al oat to semi- 
. departmentalization. + 


a ae 


we! re using Unit Leaders. c'r@ attempting mul tiage “ 
grouping, ° "he, :. ow, ae * 


Very. different, We're teaming, mu}tiaging, and children 4 op 
rare moving at their own pace. a 


We're teaming more effectively. We! re using the ie 
program, . 


J 


it 


‘| More small groups; more waitiane ‘instruct ional: groups; 
more aware of assessment needs. 


. Math is different. 


% 
: ' 


‘We're still graded now (next ‘year we'lT ungrade) but our 
teams are es ‘ 


We have wltiaging and Unit Leaders; Teachers ‘are more’ 

involved in*the totaT school program; Our group sizes vary 
‘and we're using one to one instructional situations; © 
“Older children are working with younger ones; Children 

are working at success, levels; Children’ are being assessed 
| by more thart one sabi 4 


. « The data from the bane above are 2 limited in quantity, sees very 
r ' significant.” With the exception of one school, all‘principals felt that 
hg , a definite change had taken place, although the change might have been a t 
ae ik very slight. More importantly, the data showed that IGE Ras progressed 
oN on a broken front and that future efforts at ee ould consider. pes 
‘the d Frerent levels of growth of each, school. oe A 


a es Sama Sis : 4 fo 


© Review the evaluation: procedures for each oblective. The 

description of the procedures should include who did what 
to whom, how, when, and aN ane SOE to collect* 
evaluative, data. 


- . - i ; e \ an 
Validation of Evidence: ‘ > a 


ge te eee 


a 


» 


Longitudinal -data net yet available. Recommendations from Wisconsin - «< - 
and Kettering 1/D/E/A indicate a three to five year period to implement ' 
a total IGE program. Data available at this point indicate progress is . 

as anticipated, ’ 


y . 4 ; t 3 ae : te 
%s "Evidences! _ANT data available for. pe kao 7 Pay ee 
Collection of baseline da 


a” # 


considered to be strong : 
Evidence: All data evaluatéd in sondunebtan with each of the opteceives, w 


- Monitored by external See as well. as piadeck 
personnel, 


Evidence: bpndact League, evaluation report from the_ enter ‘for 
* Administrative Studies, Sel~aveqantten report. 


‘e © “4 a i 
a \ Fe . 


v " ‘ 
; ‘ . a 6 a ‘ 
am - Rating of Evidence... . Sey =: ; ES al 
acceptable (1 point), ; : by he 4g | - 
ca eaceneenneaemed ’ 4}. ~ 7 4s, 4 ¥ k 
-)_*___ not acceptable (0 points): Explain ‘ =. 4 & - 
‘. . ‘ » ‘ ; ng q ' : ‘ 
, Also record in appfopriate box , ‘ ‘ : 
2 X ¢ c ™ 
, : { 
, ‘ : 
(p e . : 
: \ \ \ YA 
( : ca 116. « { / \ “ tt 
, | ; NS _ | f27 Y . 4 ‘ : 


* To what extent are the Senenaiiresy activities, or . 
_Methods described presently in dperation? The evidence 
.for the validation of this section will be in.the form = | 
\of written information provided by the project director. 
This will.be verified by on-site observation and tnter- 
Hews: with persons associated with the project. 


To" ‘be, completed during the on-site visit. 


i: “e » Validation of Evidence: : rae 


Bae y Av “hdgusthent io differences in ability, learning rates,. learning tyles 
% “can profitabl be adopted and/or improved:in all school distr7cts. 
‘Parental Anvolvement has provided an ‘important SRE OR Tate: to he Project. | 


Evidence: ee. records indicate esloniteay services from S conn Yt td 
i, - parents have provided a series of pamphlets related to.IGE; agendas ae 
* advisory committee meet ings are available af MEC; Newsletter. tre ge te : 
The emphasis ‘in the program has been on staff development. for. 
leadership in individualizing instruction; student population cut across 7 
‘ the entire range of learner characteristics of the-schools voluntarily ps ‘ 


* ; participating. These populations are described in the MEC records; 1. Ges 
* : prejject League. © F a: 
: 7 r, % 4 : u 

i Documented evidence’ hella at MEC? Units of teachers | ; 


( ' described, materials cataloged, inservice programs described, feedback 
from ried meetings and HUB committee sessions available, clear: 3 
) organizational. charts ae available. In addition, considerable informa ~ 
tion about IGE is. general ly available. 


| Much of the data processed by computer. | Credentials of staff — 
; wank or exceed competencies needed, &. 


Evidence: Examination of data, both descriptive and inferential for all 


objectives, : 
. ‘ Rating ‘of Evidence ' ms a “4 Jet 8 ; - 
h , ae i ia wn 
PY . ‘ ee acceptable (1 pont h, ji ey * ’ : , we A : ae 4? 
- not: acceptable (0 points): Explain eos ee, 


Also'record in appropriate bax a ae ei | ae 


i a a ’ z “ ve on vi Ay > ma : 
2 gee Se . i 4 e katona a. ’ 
f + MD, Ss, * A S # og rie MWR og hy ye : <4 N * NOS % 
| " ‘ pcan : 
a . . id. Js he ‘ Se “ft ; vege eit = 5 erm. ’ . & 
sy % Ts Ne. wee ah OH co laa % a / Re he 
i : ab fh tee 4 ahs aes. t Oe ; ; TABLE 20> i oo P , F a 
* an 3 ” ' 3 ; . , ie 
| hae e% EVALUATION SUMMARY ig a a 1 0 a 
See oe) ee “. * EVIDENCE OF EFFECTIVENESS. be : ne as 
Fee ti RR ww, g.0t4 ‘ + " ‘the i 3 to» ( ; . 
’ . j . % - ee | a “J vw : | ’ ; * - =: a ' 
@ his “TYEMS * 
» i ‘ “ 
, esi 
SL as Activities | 
| “Attainment. 
rg . ‘ Ps 8 "y 
a | ; Cause. of Attainment «. 
+d. Evaluation , 
er Continued Operation 
| Tom &L 7 + 
| ; | 
} ' *Items bs Cy a: and e must score 1 point. If any of these items ‘do not - 7 
wm » receive a rating of 1 point, reject the objective for further validation. : 
| 2 al Each objective*must score a total of 4 points for valications If SO, « eet 
eo ‘record 4 in the. appropriate box at the bottom of this page... ae 
| If the objective receives a score of le$s than 4 record a oe din the 4 a i 
4, appropri ate box, at the bottom ofthis page. . 4 \.. Rf ye 
‘Coe oer : 
| ~ a The team hhomber is to average the scores they have given all of the Me 
4 onjectivas and record it in ene box labeled average." ae P : : 
id oe ‘ . Pe 5. ’ 
: a a ok | OBJECTIVES : A eS 
’ es i ae i d ~ A ae : s : Tua y 
| if : Brey ‘ : - a : . * ? fs, . ee &) {2 ra es 
oe ae 8 ak. Re 3 4: 5 Average © 4 is 
f 


\ 


The hae ee éeldenew ot ‘the ongoing 


~ Ex xportabiity - He” pO al ee 


‘To what extent are the srocadurere: activities, or netitods described g a 
presently in operation: The evjdence for the validation of this section 
“will be?in the. form of written information provided by the project we . 
director. ‘This Will be verified by on-site observation and interviews 
"with persons, associated with the project. 


\ \ 
S ; “a % 


“¢@ Examine and verify the evidence seaelias for, ach iia: 
‘weported in the application for validation, mARGR CARTIER 


) “Assign on:the scale below,a point value of myn if you feel 


© the. program should not. be validated, ~Include recommendations. ‘ta 


® Assigt a point value of "2" on the “scale below if. you feel” “ 
the @rogram should be verified for state informational 


dissemination. Include recommendations, ae, At, 


e Assign a point value of "3" on the scale helow if you fee} 
you: cannot certify it for state diffusion ‘but with the ~ 

- adoption of some recommendations validated for such certi- 
fication may result. Include recommendations. * i 


the project is worthy of State ATE fus Ione 


Record the score Jabove. / % ‘ 


‘Recommendations 


g* 


_© Biven ‘the condi ton of similar interest and leadership, the Project pro--. 


vides material for replication. Available at MEC are:, needs assessment 
instrument, inservice materials, organizational descriptions, brochures, 


Sand booklets on such topics as learning Bie les ang skills SNCCRINGES 


newsletter. | 


process dnertngti 
to the Project is available and valuable for the replication of the © « 
Project. Interim ‘evaluations, peer process: records, minutes Of, princi- 
pals‘ and HUB meetings provide ‘data relative to procebss ©” 


| Z a, . 
i | oa eo 


y ; \ 1 
hi ‘ 


e@ Assign a soln value of "4" on the scale below if you fe9) ; Las 


i 


ie. the actual ( antetaatt wakesnnak is» baste to the aicive. prodese” ates 


“. . 4t is. particularly unrestrained by any. need for numbers of highly. i ; " 
‘ " trafned specialists. Staff development. ig yrovided as an essential 
r focus of the program. - aise ‘ 3 = mea Res Be 
“Methods of reproduction of materials’ ‘are essentialIy\ ose ta e mn a 
the ma ority of soho districts. Be, f 


Since most ate cronal programs utt11ze\commerctally aia patartals 
o in addition to locally produced materials, cost cannot be epistreed ass Nf Ss 
Setiaening from potential for adoption. : 3 , es 


s 


About three dollars, per Pups were added to routine cost of instructional 0 
_ materials. in Be oe * 


A district seriousty iiikaraubad: in "raking the ‘changes \edeeited by the :. oe, 
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submitted in. advance. , e 


- I think I would have offered only ‘the morning or afternoon programs. 
This program was too congested for one day. I think that a follow-up 
asking auenevans should be held next session with good and bad questions 
pr sente ‘ ' ; 


ny Z A cS ; ‘ 3 z : L i 
= ti sine eT » ‘ Fi . | { ‘ r ; . ; ° yar é e 
ar - Wire, on tial, for-inst&nce:: ‘small groups with the task of || - 
, * evaluating a hypothetical project, or designing evaluation component 
; ‘for anew project. I-would'be more clear and direct about my Z 
eae, avai thg Questionnaire and getting input from participants. ; 
> Not enough ‘time to attend aul of the seminars I was ‘interested’ Ws ale iA 
= OK. : PO ER oceans Agee ) é 
Te Yes, I thought it was. good. - 7 > we” 
. t= Not, sure a et ; hae ee 


= Good format, »but needed’ more bite. I would have liked to attend all ny: 
"three sessions. Very interesting, Aa bi Workshop leaders “ ; 
‘Seemed under time pressure to get finished, ae Rett ur ar ae 


* «= Critique individual project evaluation plans.” * 
- More depth, less choi¢e on "exposure" . , 3 she Or A, iy ee 
+» Because we operated’ on sich ‘ “tight time dices oS. | 


' 9 More time. a 
2. & et i cs tas 
2 * or et = 
wo mes Training j % . . : 
a u haye any. _euggeotione Foe future training? © Plea epect fy wt 4 
' (topies, apeakere, demonstragiong)., ; *. ft . 
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Do you -think there should be fetuses trainin. panaidia? Pg 
mbigdtt® a, ve : 2 __2_Rropthion/io Sure os : _=1_No Response’ i 

\ Commente: . ; ‘ 
-"I feel that I could: not work’ on skills ina practice session which 


- would have been helpfci. 


i Specific skills. Take time to design your own jiccboanniten: Then 
get reactions and rebuild. : v. ym 8 4S 


- Not at this time. = re | ee 


-.1, More on funding options — ; 
2. Working with School Conmittées © °. “ 


Dem, 
Wey Ah ® The Educational Research Corporation Program is. ‘good, no excel ent.” a ae 
“ ' ** “J would endérse and encourage it. ee 


. Continue present approach. a 
- Grouping of participants: who are in attitudinal Danis ak evaluation ee 
for techniques and Strategies. . e.: 

% ‘Hold some ‘in kestecn ‘Mass.! ‘ 
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"and how wé can use each other as resources over and above sharing 
information at these, conferences, 
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at APPENDIX E 


. * td ee ‘ ~~ t : % . 
Z - mete) dl “USING STANBARDIZED Tests ue 
¥ : ’ ; . % Me “¢ a . 


La w 


: You‘ ianyss ‘to use one of the several standardized tests to pretest 


veneer “rand post-test learners’ and comparison groups- if your’’program. ts attempting “mer 
to improve achievement. Frequently such tests are a part of the regular . = , 
schoo ‘testing program . anny $0 your use of these tests in your Title ie 


- Achievement by Horst, “ i 
. @iscysstons about: the: atrecte” that" each. "ot the. following Ppoints. have, in ' 
. creating val fy ernlention ae : = 


¢ ‘Always test students within a few weeks of the dates ° 


innoyative’ program would convénjent. You are, probably familiar with . 
the standardized testing process, but’since: the\tests are so commonly * 
misused it may be worthwhile for you to- ft BENS we of these 
common ° misuses. ee 

For complete * explanations of. sig ‘ade 

please refer to:. ee ue 2. “ pasar ee " : 


q 


1, Do not use.  aradenocuivaleat scores for computation or ot. 
- veporting since grade-equivalent scores are projected * 
figures in many cases and: dre, therefore, misteading. w. 
Convert raw. scores: to standard*scores for computati 
_ + «> Means can then be converted ‘to perceg eile oe 
for reporting. ; ¢ 


ss 2." Never use gain scores to try to adjust - for ifferences 
.' + 4n scores among treatment and comparison 4 oups because: 
use of gain scores cannot adequately do this., 


on which the normed groups were tested, Regardless \ 
ry 2. Of what test Teal ated might claim, normative data *..4: 
‘ are‘only really applicable when students and the oii. < 


> 


(s+, population are tested ena the same time of the 


yefr. ay * 


a 
me ose the appropriate level of test. * students should” 
ore in the mid range of possible raw scores, slightly 
low the mid range.on the pretest slightly above on 
"the post-test. Always use ‘the same Tavel of test for. 
ue and POREA EASES by \\ % 


+ 
aaa 


our‘ utmost to get retests and perecteats for all 
aa icipants but make pretest/post-test comparisons” 
~ only for Res students wi th both sets of.scores. 
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6.. inister and score. tests carefully. Use orderly. Oe 2g 
st procedures and follow publishers directions a 
* " sccurately, Test al] students under exactly the same =>, ew 
conditions for’ both pretesting and post-testing, this 6 6) ej: 
7 -» - facludes same time of day.and same place for testing, °°. 2° ° : 
sretog stan i es cael a ‘Use: onprogram people for. testing if. possible... adh ents oh wind nano 
; a If achievement. gains appear do not assume they resulted’ ig Egon 
; , .. ' from"your project. Examine all possible alternative. to ee Sr 
Ps \. + explanations. This may require applying different ° ee a 
: + a of control ‘procedures .in succeeding years before. 
_ all baaslah ig © explanations of. gaing can be slasinatet, 


* s* 
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re ‘" 7% 
5 a 8 If, project. participants’ tre sata ause.they " 
, na - scored at the high end of. low end on a standardized * aa a 
a test, you will have to use special regression models va -% 
es a pe 4 for analyzing. your test data,~— Pn meee” er ee A 
} here are numerous: standardized achievemen ‘tests; {n use: today but. below |: a 
am “list of, the more commonly used test batteries, the general mniesy yo ae 
ss r areas tested, ene the grade Wives covered, in the tests. ee 7 eS 
es oe a = ap i ae oe 588 ag St te Me 
ou + Achievement + Pe, Mahe Subject Areas ; \ 4 e Grade. . -. 


Batterics eo 7. . ‘2 Tested ae 


Com rehansive Test oF; - ‘te eading, * Lahguag e Arts - . 
‘Basic Skills (CTBS) : * Math, Study ‘Ski Is ee ye “2. slag ae 


“ 
Pele ton of the California t Stenting Language. Arts, . 
fe .% PS | Test (CAT). ‘Math, Study pers : * 5-8. ° 


‘Jowa Test; Of Basic - ~ ? Reading Lariquage Arts: a 
Skills: 4178s) _ *Math,. Work Study, Pt ar gre 9 " 6 


‘vy "+ Jowa Test of # Ao +» Soctal Concepts, ‘Natural oe “12 
~ Educational cs Sciences, In rpretation, Stee Et 
iy Bey ie lermnit Jsitepy™ »’ .» » $Soctal Studies\ Literary » os it 
oc a ee vi: ee ‘Materials, Vocabulary, =” Ss 
ss . ge ahs tm Satine” Use of Materials eis Se es ig 
_| Metropolitan Achtevenent % ‘ Reading; Language Arts ae 1.5+ 8 
¢ 7 9 eg (MAT) i 7 i ‘ : . : yore 

2 tap ohnes . ‘ WAL ai» Po a Z 7 “} vi soc above. pluse: =. ah : ‘ 
_ oH By Social Studies, Science, va. ae 
, 7 Study Skills . ee a 

“* The above plus’ Math Analysis Be gp at 

3 ¥ and Problem Solving, oe rsot tte : 
fe uv oe oa Concepts and Understanding. ” 2% - 1 
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see _ Sequential Tests of: Educational | Reading,- Writing,” Math, 67° 4-12 Te 
- Progress (STEP) - Science; Social Studies, ok of ee Se 
| mt Pe as Me “6 en o oe, a ty 


i er | “A weet ; 
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% “eg R ' . a .: pnt rae 
4 "s é / 4 - t oh = 

is a oe < uf *; "7 whe e Ee ag 
. : # a é . : y a he % . 
ae ‘ a General : ; he - : : at 
a. ‘fatteties - Py $ ject Areas —_ 3 ” 

a ee Batteries ‘ s\ ~mested “y-  ®. 
aE ‘ Maha 7 : p. te e cy ' ae a 
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Ty sans Ach{everont Series ' beading, inpaee Arts; 
ne ry he aM ath k Social Studies ; 
ie.” & re Mae a ail ‘ 


el. ibs arts : 
nat 3 Standard citer n Ow ng, Sp Hing, Math cs ¥.5 219.9 ey 
Peni ‘Tests (SAT) ey. ao % ps ; 


% ne > Pua. Slap ‘ ' 1§ Sdciat Studies, Science oe Grades , a 
© ip Par ao gs 4 , Fi . 
pie Bea!  StanWrd Achievenent a Engl tsh,, Math “Reading, a, BY aye 
fe Fog CORE” '¥., Sclenc& Social Spudies.* | 9 a | 
af’ ‘ $y : . ‘ © 4. # x" os 
if a oo Test of Keattate ‘igen Sect. Sty ia ompdsition, er: es a og 
oe ns ERS eae . Science, fReading, pees Fag Bs ode ee 
Mea Pi. ’ i” Literatyre rd a ea: 
‘¢ 


fae 
seg 


September 22 - Project Directot's Orientation 1 Meoting, fie; Title Iv-G ae - 
Wednesday =~ ae = ; a -  . staff Ds 8 


October .5 ,  ‘Rolé of: Project’ Director . i rn ie: MEC ae , 
, Tuesday’ - .” \. - Problems/Issues ae - 3". Ron Havelock ‘ ® 
Oe dae ms re cP = Case Studies # : Pg yt : . 4 
5 eo G3 _ Problem Diagnosi8/Needs kauneusent ee? 
(4 + =. Skille nekded for aefestars local seesecenty BoB ae 
October 28 Simulation Gaine’ ia 
rsday Dissemination ges ae 
: , ! 
2 a ' v 
” November | 7 Leader ship /Manag ent ‘Appraisal : a ‘ MEC «Avian llis l 
Nadayelley _*"  In-basket action. exercise j i a of Educational .* . 
‘+ = Simulation!) . 75 UN,’ Resdarch COFP waren ! 
Dea 20 Se "Neonates Change 5 ae : ey - o _ MDE -." gg ft 
Pasian: »-  .,— Models . ' 5 a a Ae ae 
> t 3% Utilizing Resources . y  :MEC & Havelock . 
7 a “coping Strategies . s “. hes es i 
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February Analyzing and Impro ing Program ; : .  MEC & Glen Heathers” — 
Tuesday : simp lesenystion: a ee -of Research for  —=—>—y 
' a 2 oa ; "|. Better Schools _ or 
‘ mg z Ree hk, Ge eens 4 ‘a 
March 14 Py Implementation’ Strategies = .. | hake @ MDE, ee ; 
Monday 4° “--" )-»=-Cénsumer-céntered ; ME¢ A Heathers t o> 


oe A ~Product-cénteres change. str regis 


art : i camer aur os : . 
Aptil 13. To.be announced eg a: rr ae.) ene Gears | 
Wednesday Ge ce i oe he > (f | * 


May oo ot ee Informal Comniiti cation is | aa ; is t : 
Monday. . —s. = - Political Behavior _ r, * same HEC: & ; 
ee be >" Political Implications | * a ' sk Butts 


; _ May 26, ee ' Networking ees Me om, € : “, MEC re velock 
. seen - Dissemination Process - ~- . 
By ~ Communication Coptatite & reports) Fi hs 
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Further Headings in Evaluation 4 oF if ee i, a4 i 
_ a 8 at seg ae ahs wit 
Andgrson, seaiiehs B., Samuel Ball; ‘cine: T. Morphy and. Associated « be ae 
oe _ Encyclopedia of Educational “Evaluation? ‘San Francisco, CA.” te 
“} “, . ae ; i 9 ae, tee, ; * : Aho ; : 
~ tee ; ~ Be ge , 
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; _ es ; “Use ‘the most recent edition available for a complete listing| 2 96" A 
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oe oe Popham, W 3. (Ed.) tvdluation in Education, Barret Heclichan ete | 
> ae / Publishing Corp. s* 1974, ‘ ‘ Le $, . 4 ss 
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: to _Teebrin, T.D. ! Evaluation: cA practical Guide for: Teachers. i \ 
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_ Bloom, B.S. J. Thomas HasttnossPaed George F. “Handbook on ‘ 
_', Formative an Suna tive valuation of Studer Learn n ae oF eS 


: bie BuSK. Qa. ) Taxonomy Se Educational Ob ankiven The Classification ' 
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